Abstract
We can adapt flexibly to environment changes and search for the most appropriate rule to a context. The orbital prefrontal cortex (PFo) has been associated with decision making, rule generation and maintenance, and more generally has been considered important for behavioral flexibility. To better understand the neural mechanisms underlying the flexible behavior, we studied the ability to generate a switching signal in monkey PFo when a strategy is changed. In the strategy task, we used a visual cue to instruct two male rhesus monkeys either to repeat their most recent choice (i.e., stay strategy) or to change it (i.e., shift strategy). To identify the strategy switching-related signal, we compared nonswitch and switch trials, which cued the same or a different strategy from the previous trial, respectively. We found that the switching-related signal emerged during the cue presentation and it was combined with the strategy signal in a subpopulation of cells. Moreover, the error analysis showed that the activity of the switch-related cells reflected whether the monkeys erroneously switched or not the strategy, rather than what was required for that trial. The function of the switching signal could be to prompt the use of different strategies when older strategies are no longer appropriate, conferring the ability to adapt flexibly to environmental changes. In our task, the switching signal might contribute to the implementation of the strategy cued, overcoming potential interference effects from the strategy previously cued. Our results support the idea that ascribes to PFo an important role for behavioral flexibility.
SIGNIFICANCE STATEMENT We can flexibly adapt our behavior to a changing environment. One of the prefrontal areas traditionally associated with the ability to adapt to new contingencies is the orbital prefrontal cortex (PFo). We analyzed the switching related activity using a strategy task in which two rhesus monkeys were instructed by a visual cue either to repeat or change their most recent choice, respectively using a stay or a shift strategy. We found that PFo neurons were modulated by the strategy switching signal, pointing to the importance of PFo in behavioral flexibility by generating control over the switching of strategies.
Introduction
Abstract rules and strategies are important to make the correct choices that are appropriate to the specific context. Although many brain areas contribute to the generation and maintenance of rules and strategies, a prominent role has been assigned to dorsolateral prefrontal cortex (PFdl) from many studies (Milner, 1963; Collins et al., 1998; White and Wise, 1999; Hoshi et al., 2000; Stuss et al., 2000; Monchi et al., 2001; Wallis et al., 2001a; Bunge et al., 2003; Genovesio et al., 2005, 2008; Mansouri et al., 2006; Buckley et al., 2009). In orbital prefrontal cortex (PFo), on the other hand, emphasis has been placed more on expected or received reward (Thorpe et al., 1983; Tremblay and Schultz, 1999; Hikosaka and Watanabe, 2000; Ichihara-Takeda and Funahashi, 2006; Padoa-Schioppa and Assad, 2006, 2008; Simmons and Richmond, 2008; Kennerley et al., 2009; Bouret and Richmond, 2010; Strait et al., 2014; O'Neill and Schultz, 2018). It has been shown that PFo neurons can encode reward either regardless of the context, both as a cue and as feedback, or only when it serves as a feedback (Tsujimoto et al., 2012). Reward as a feedback has been also shown to be task-dependent (Luk and Wallis, 2013). However, signals related to rules and strategies have been described also in PFo, in addition to PFdl (Wallis et al., 2001a; Wallis and Miller, 2003; Buckley et al., 2009; Yamada et al., 2010; Tsujimoto et al., 2011; Sleezer et al., 2016).
Flexibility to adapt to external or internal changes has been studied with a variety of tasks requiring different forms of behavioral switching such as set shifting, rule shifting and reversal learning. One prefrontal area considered important for flexibility is PFo based mainly on the use of reversal learning paradigms. Reversal learning deficits after PFo damage have been reported in marmosets (Dias et al., 1996; Clarke et al., 2007) and humans (Rolls et al., 1994; Fellows and Farah, 2003; Hornak et al., 2004; Kringelbach and Rolls, 2004; Tsuchida et al., 2010).
While such deficit has been found after the PFo damage also in earlier studies in macaques (Butter, 1969; Iversen and Mishkin, 1970; Izquierdo et al., 2004; Rudebeck and Murray, 2008), more recent ones, with lesions sparing the fibers, failed to find a deficit (Kazama and Bachevalier, 2009; Rudebeck et al., 2011, 2013). In addition to the lesion studies, also a recent fMRI study in macaques addressed the role of PFo in task switching reporting activations during switching (Premereur et al., 2018).
In this work we aim to test whether the switching signal, which can support flexibility, is present in PFo with a neurophysiological approach and with a task design different from the reversal paradigms because in our paradigm task switching is not dependent on previous errors. To examine the role of PFo in switching strategy, we have used a visually cued strategy task described in prior studies (Tsujimoto et al., 2009, 2011, 2012; Tsujimoto and Genovesio, 2017; Fascianelli et al., 2019). We studied the behavioral switch between two strategies (stay and shift) each cued by different visual stimuli. The switch between strategies was required by cuing a different strategy and not as a consequence of an error signal as in reversal learning paradigms. We have previously shown that PFo neurons convey strategy signal and that this signal develops earlier than in PFdl (Tsujimoto et al., 2011). Here, we study PFo single-neuron activity to test whether PFo neurons signal the strategy switch and could then have the potential to contribute to adapt to environmental changes. Compared with previous studies (Kamigaki et al., 2009; Sleezer et al., 2016), our paradigm allows the dissociation of switching from outcome signals because the behavioral switch, by task design, did not depend on the previous trial outcome: unlike tasks used in previous studies, both switch and nonswitch trials could follow correctly executed and rewarded trials.
Materials and Methods
Subjects.
Two male rhesus monkeys (Macaca mulatta), 10–11 kg in weight, were trained to perform a visually cued strategy task before starting the recording sessions. Each monkey sat in a primate chair during the execution of the task, with the head fixed and oriented toward a video monitor 32 cm away. All procedures were in line with the Guide for the Care and Use of Laboratory Animals and were approved by the National Institute of Mental Health Animal Care and Use Committee.
Behavioral task.
A sequence of the task events of the visually cued strategy task is shown in Figure 1A (Tsujimoto et al., 2010, 2011, 2012). In each trial, the monkeys were required to make a saccade toward one of the two spatial targets, according to a shift or stay strategy cued by a visual instruction (Fig. 1B). The appearance of a fixation point (a 0.6° filled white circle) located at the center of the video screen, with two peripheral targets (2.0° unfilled white squares) placed 11.6° to the left and right of the fixation point, represented the beginning of a trial. The monkeys had to maintain fixation on the central spot for 1.5 s; after that, a cue period of 0.5 s followed. During the cue period, a visual cue appeared at the fixation point. In each trial, one cue was chosen pseudorandomly from a set of 4 visual cues: a vertical or horizontal rectangle (light gray) with the same dimensions (1.0° × 4.9°) and brightness or a yellow or purple square with the same size (2.0° × 2.0°) (Fig. 1B). A “stay” strategy was instructed by the vertical rectangle and the yellow square. If a “stay” strategy was instructed, the monkeys had to make a saccade to the same target chosen in the previous trial. Conversely, the horizontal rectangle and the purple square instructed a “shift” strategy, for which the monkeys had to make a saccade toward the target that was not chosen in the previous trial. The end of one trial and the beginning of the next one were separated by an intertrial interval of 1 s. The monkeys had to maintain fixation on the center of the screen during the whole fixation period (1.5 s), during the cue period (0.5 s), and during a subsequent delay period of 1.0, 1.25, or 1.5 s, pseudorandomly selected. The fixation window was a ±3° square area centered on the fixation point. Both monkeys maintained fixation accurately and rarely made a saccade within the fixation window (Tsujimoto et al., 2009). Any fixation breaks during the fixation, cue or delay periods led to abortion of the trial.
The fixation point and the two peripheral targets were kept on the screen for the whole duration of the delay period after the cue. The monkeys were required to make a saccadic eye movement toward one of the two spatial targets after the disappearance of the fixation spot. The monkeys had to maintain the fixation to one of the targets (±3.75°) and then both squares became solid white. The capture of the gaze into the response window represented the “target acquisition”. The monkeys had to maintain the fixation on the target for 0.5 s (prefeedback period). Any fixation breaks during the prefeedback period led to abortion of the trial. After the prefeedback period, in the case of correct responses, feedback was provided as a liquid reward (a 0.2 ml drop of fluid) or, in case of incorrect responses, as red squares over both targets. After an error trial, the same visual cue was presented again in the following trial, called “correction trial”. Correction trials were presented until the monkeys responded correctly. Usually, after an error, not more than one correction trial was required by the monkeys (Tsujimoto et al., 2009).
Data collection.
We used up to 16 platinum iridium electrodes (0.5–1.5 MΩ at 1 kHz) inserted into the cortex with a multielectrode drive (Thomas Recording) to record single-cell activity from PFo. We positioned and angled the recording chamber (18 mm inner diameter) according to magnetic resonance images (MRI). We isolated single-cell potentials off-line (Off Line Sorter, Plexon), based on the multiple criteria, including principal component analysis, the minimal interspike intervals, and close visual inspection of the entire waveforms for each cell. Eye position was recorded with an infrared oculometer (Arrington Research).
Histology.
We used standard histological analysis and MRI to reconstruct the recording sites (Fig. 1C,D). Approaching the end of data collection, electrolytic lesions (20 μA for 20 s, anodal current) were placed in selected locations at two depths per penetration. The animal was deeply anesthetized after 10 d and then it was perfused with 10% (v/v) formol saline. Frozen, coronal sections were Nissl stained. PFo recording sites differed slightly between monkeys. In Monkey 1, they were mainly from area 11 and ventral area 12, more lateral and rostral than in Monkey 2, which were mainly from area 13. The locations of recording sites have been illustrated in more details in Tsujimoto et al. (2011).
Data analysis.
We analyzed the activity of single neurons in PFo (Tsujimoto et al., 2009, 2011, 2012). We discarded correction and error trials from the analysis. We decided to discard the few correction trials because in these trials the monkeys in principle could perform the task without integrating the information of the cued strategy with the previous spatial response. Indeed, after an error, the monkeys could just change the spatial response with respect to the previous (incorrect) response without following the cued strategy. It would not be necessary to follow any strategy to perform well in a correction trial.
We defined the task periods for the analysis according to time windows aligned to task events: the fixation period (−500:−50 ms before cue onset) and the cue period (50:500 ms after cue onset).
The cues instructed the monkeys to follow one of the two strategies: to stay with their most recent successful response or to shift to the alternative response. Based on the strategy cue, we designed each trial as stay or shift trial.
Moreover, we designed nonswitch and switch trials as follows: in nonswitch trials the instructed strategy was the same as that on the previous trial, while in switch trials the strategy differed from that on the previous trial (Table 1). From Table 1, it can be seen that both the nonswitch and switch conditions are not associated to specific visual cues. For example, the same vertical rectangle cue can be presented both in a nonswitch and in a switch condition.
Definition of the nonswitch and switch conditions between two consecutive trials
Along the article, we refer to the signal related to the switching of the strategy between trials as strategy switching signal or, simply, switching signal.
Only neurons satisfying the following requirements were identified as task related and were analyzed: (1) at least 1 Hz of mean activity during the cue period; (2) at least 10 trials per nonswitch/switch conditions; and (3) mean activity during the cue period significantly different from that during the fixation period (t test, p < 0.05).
The analysis was performed using MATLAB with Statistics Toolbox Release 2015b (The MathWorks).
To investigate the encoding of the nonswitch/switch conditions for the task-related neurons, we applied a one-way ANOVA (p < 0.05) to the cue-period activity, with nonswitch and switch conditions as factor. As a control, we performed a permutation test in the cue period obtained from 1000 iterations. For each iteration, we shuffled the conditions (nonswitch and switch) and we computed the one-way ANOVA for the shuffled data to obtain the percentage of significant cells. The significance cutoff was assessed at 95th percentile of the distribution of 1000 iterations. We also quantified the strength of the strategy switching signal by performing a receiver operating characteristic (ROC) analysis. ROC values reflect the ability to decode a signal based on the activity of single trials (Dayan and Abbott, 2001). We defined the ROC values as the area under the ROC curve (auROC). The auROC values range from 0 to 1, where 0 and 1 indicate the maximum selectivity for the opposing preferences. The normalized values of auROC were computed with respect to the preferred condition—that is, highest activity—and it ranges from 0.5 (no selectivity) to 1 (maximum selectivity). We estimated the time course of the switching selectivity during the cue period using a sliding ROC analysis for the nonswitch and switch conditions. We computed the auROC in a 200 ms centered window that stepped across the trial in increments of 5 ms. To assess the independence of the strategy switching signal from specific strategies, we divided the trials by strategy. For each neuron, we computed the auROC for nonswitch and switch conditions, separately for stay and shift trials. Each stay trial was classified as nonswitch (or switch) trial if the previous strategy was a stay (or shift). Each shift trial was classified as nonswitch (or switch) trial if the previous strategy was shift (or stay). We then computed the correlation coefficient between the auROC values (for the strategy switching coding) for stay trials and shift trials. A significant correlation (Pearson's coefficient, p < 0.05) would indicate that strategy switching modulates the neural activity similarly for the stay/shift strategies.
Moreover, to investigate the strength of the switching signal and its dynamics at the level of the whole population (task-related neurons), we performed a neuron-dropping analysis using for classification a peristimulus time histogram (PSTH) classification method (Foffani and Moxon, 2004). For each neuron, we split the trials according to the nonswitch and switch conditions. We considered the time period from 500 ms before the cue onset up to 500 ms after the cue offset as time period of analysis. We used a time bin of 200 ms stepped by 50 ms. For each time bin and for each neuron, we randomly picked one trial from the same condition (test trial) and we calculated a look-up table of the mean firing rate of all the remaining trials (one test trial held-out) for each condition. The test trials were assigned to the condition with the lowest sum of Euclidean distances between each test trial and its corresponding neuron's look-up table. We repeated this procedure 1000 times for each condition. We then performed a neuron-dropping analysis. The neuron dropping consisted in repeating all the previous decoding iterations each time by removing one neuron from the whole population, until the population was composed by a small group of neurons. In our case the smallest sample size was of 10 neurons.
To assess the involvement of the switching signal in the behavior, we performed an error trials analysis. We looked for a relationship between the switching activity and the behavioral performance by investigating the maintenance of the preference—for the nonswitch or switch condition—between correct and error trials. Any change of preference between correct and error trials would suggest that this signal could play a role in the task performance. To answer this question, we applied a bootstrap analysis as in Tsujimoto et al. (2011). As first step, we identified, for each of the significant switching neurons, the preferred condition (nonswitch or switch) for the correct trials, comparing the mean activity across trials in the cue period. We then assigned the preferred condition, assessed in the correct trials, to the error trials, for each neuron. For instance, if a neuron preferred the switch condition in correct trials, then in error trials the switch condition was addressed as “preferred”, regardless of the activity on those trials. For each neuron, we computed the difference between preferred and anti-preferred activity for correct and error trials separately, and we averaged these differences across neurons, again for correct and error trials separately. If both differences were positive (in correct trials the average difference was always positive by construction), it would indicate an average maintenance of the preference between correct and error trials, not suggesting any relationship between the switching signal and behavioral performance. On the contrary, if the average difference for error trials was negative, it would indicate an effect of the switching signal on the behavioral performance in which the switching activity followed the monkey's erroneous behavior and not the correct behavior requested for that trial. To assess the statistical significance of the two differences (to rule out that the same result could be obtained by a random combination of trials), we randomly shuffled the switching strategy labels (nonswitch or switch) for correct and error trials, separately. We reassessed the preferred and anti-preferred conditions for both correct and error trials using the same criterion as for the observed data. We repeated the same analysis as above for the shuffled data, by computing the mean activity difference for preferred and anti-preferred conditions separately for correct and error trials for each neuron, and we then averaged it across neurons. We repeated the shuffling procedure 1000 times, which yielded a distribution of 1000 sets of activity differences, to which we compared the observed mean activity difference.
Results
In the strategy task (Fig. 1A), after a fixation period in which the monkeys were required to maintain fixation on a central spot, a visual cue was presented (Fig. 1B) instructing the monkeys either to “stay” with the target chosen on the previous trial or to “shift” to a different target. After a delay period following the cue period, a go signal instructed the monkeys to make a saccade either to the left or the right target. In each trial, the current strategy had the same probability to be repeated (nonswitch trials) or changed (switch trials) with respect to the strategy instructed in the previous trial. The monkeys were required to maintain fixation on the chosen target until the feedback time.
Behavioral task, cues, recording sites, and schematic of the nonswitch and switch conditions. A, Temporal ordering of task events from left to right of the visually cued strategy task. The dark gray rectangles are the video screen as viewed by the monkey. The target of the monkey's gaze is indicated by dashed lines. B, Visual strategy cues presented to the monkey. Each different color and shape instructed the strategy to be applied. C, D, Recording area for PFo (C) and PFdl (D). Fb, Feedback; LOS, lateral orbital sulcus; MOS, medial orbital sulcus; AS, arcuate sulcus; PS, principal sulcus. E, Schematic of the nonswitch and switch conditions for six consecutive trials. In each trial, the cue and the following saccade to the spatial target are shown according to the strategy instructed by the cue, i.e., stay or shift. Trials are classified as nonswitch (trials N + 1, N + 3) and switch (N + 2, N + 4), respectively after the repetition of the same strategy and after its change.
Figure 1E shows a schematic of the nonswitch and switch conditions for six consecutive trials as defined in Table 1. For example, trial N+1 in Figure 1E was a nonswitch trial because it required a stay strategy (cued by the yellow square) that followed a previous trial (N) with a stay strategy (cued by the vertical rectangle). Trial N+2 was, instead, a switch trial because it instructed a shift strategy (cued by the purple square) that followed a previous trial (N+1) with a stay strategy (cued by the yellow square). In both cases, the nonswitch and switch conditions were independent from the visual cues presented in each trial, as stressed in Figure 1E (also see Table 1). In general, we refer to the signal of the switch of the strategy, as strategy switching signal or, simply, switching signal.
Behavioral results
The behavioral results have been presented in the previous papers (Tsujimoto et al., 2009, 2012). Here we focus our analysis on the nonswitch and switch conditions. We found that the proportion of correct responses in nonswitch and switch trials was on average 92% for Monkey 1 (binomial test, p < 10−6), and 92% for Monkey 2 (binomial test, p < 10−6). This result shows that both monkeys could switch or not the strategy when appropriate.
In addition, to examine whether the nonswitch trials were easier than the switch trials, we compared the behavioral performance between nonswitch and switch trials. Mean correct response were 91% and 93% in nonswitch and switch trials, respectively, in Monkey 1 (χ2 = 5.65; p = 0.02). Mean scores were 92% in both nonswitch and switch trials in Monkey 2 (χ2 = 0.43; p = 0.51). These results indicate that there was no switch cost for both monkeys. One monkey, on the opposite, showed a very small advantage in the switch trial (less errors) but the difference with the performance on the nonswitch trials is too small to be informative. Moreover, we checked for each recording sessions, and for each monkey, the percentage of trials for each response combined with the strategy instructed in the current trial (N), and with the strategy instructed in the previous trial (N-1). We report the percentage of correct trials averaged across sessions in Figure 2 for each monkey. The results show a well balanced use of the instructed strategy (in conjunction with the other task variables) in each monkey.
Average percentage of correct trials across sessions for Monkey 1 (2). The performances in the nonswitch conditions are shown within rectangles. The “X” symbol indicates not possible combinations of strategy/response of two consecutive correct trials by task design.
We compared the reaction times (RTs)—defined as the time difference between the go signal and target acquisition—between nonswitch and switch conditions. In Monkey 1, there was no significant difference between mean RTs in nonswitch (300 ± 53 ms) and switch (302 ± 62 ms) trials (t test, t = 0.29, p = 0.77). Similarly, no significant difference was observed for Monkey 2 between nonswitch (307 ± 38 ms) and switch (306 ± 36 ms) RTs (t test, t = 0.70, p = 0.48).
Neuronal results
Our neural database consisted of 377 PFo neurons (268 and 109 neurons from Monkey 1 and Monkey 2, respectively) recorded during the visually cued strategy task. The same dataset has been used in previous studies (Tsujimoto et al., 2011, 2012; Fascianelli et al., 2019). Of these 377 neurons, 33% (n = 124) were identified as task-related neurons according to the screening criteria (see Materials and Methods) and used for the following analyses.
Figures 3, A and B, illustrate the activity of two task-related neurons with a significant switching selectivity during the cue period (one-way ANOVA, p < 0.05). The neuron in Figure 3A shows higher cue period activity in the switch than in the nonswitch condition (one-way ANOVA, p = 0.004). This difference started at ∼250 ms after cue onset and lasted for almost all the cue period. Figure 3B shows an example of another type of neuron, one with a preference for the nonswitch condition (one-way ANOVA, p = 0.001). In this neuron, the switching selectivity extended also to the initial part of the delay period.
Two example neurons with switching selectivity. A, Mean activity aligned on cue onset. Raster plots show the spike times for each trial. The red and blue color represent the nonswitch and switch condition, respectively. This neuron shows higher activity in the cue period for the switch condition. At the bottom, the auROC computed for the two conditions in which 0.5 corresponds to chance level. B, Neuron with a preference for the nonswitch condition. sp/s, spikes per second.
According to the one-way ANOVA, 12% of the task related neurons (15/124) (11% and 14%, respectively, in Monkey 1 and Monkey 2) were modulated in the cue period by the strategy switching (switching neurons). These proportions of neurons were not significantly different between the two monkeys (χ2 [1, N = 15] = 0.006; p = 0.9). Performing a permutation test (see Materials and Methods), we confirmed that the observed proportion of switching neurons in the cue period was significantly different from shuffled data (p = 0.002). We have previously shown (Tsujimoto et al., 2011, 2012) that PFo neurons encode stay and shift strategies in the cue period (strategy neurons). Here we wanted to examine whether strategy cells were also modulated by the strategy switching. We performed, on the task-related cells, a one-way ANOVA with stay and shift strategy as factor to identify the strategy cells. We found 39/124 (31%) strategy neurons in the cue period. The proportion of switching neurons was significantly higher in the strategy (9/39, 23%) than in the nonstrategy population (6/85, 7%) (χ2 [1, N = 124] = 5.03; p = 0.02), indicating a relationship between strategy coding and strategy switching modulation. We found that 5 strategy cells (of 9) showed higher activity for the stay strategy, while the remaining 4 neurons showed higher activity for the shift strategy, indicating that switching activity could occur for neurons selective for both strategies. Both neurons in Figure 3 are representative of this conjunction of effects, showing strategy coding in addition to switching modulation; in particular, both of these neurons showed higher activity for the stay strategy.
For the population of switching neurons, we further explored the possibility of a dependence of the switching selectivity on the strategy type. We divided the trials according to the strategy of the current trial (stay or shift) and, for each group of trials, we calculated separately the auROC for the nonswitch/switch conditions during the cue period. We examined the correlation between the two distributions of auROC computed for each neuron for stay and shift trials separately (Fig. 4), and we observed a significant correlation (Pearson's coefficient, r(15) = 0.69, p = 0.004). This result indicates that the strategy switching modulated the neurons similarly for stay and shift trials.
Scatter plot between the auROC for the switching signal for trials with stay (x-axis) and shift (y-axis) strategy. The auROC values are not normalized, ranging from 0 to 1. Each point in the plot is representative of one neuron (for a total of 15 switching neurons). On the top left side, the Pearson's coefficient (r) with its p-value is shown to indicate a significant correlation between the two measures.
We also investigated the presence of interaction between the strategy and the switching signal. To assess the number of neurons with a significant interaction term, we applied a two-way ANOVA (p < 0.05) to each of the 15 switching neurons. We used as independent variables the strategy (stay/shift) and the switching (nonswitch/switch) conditions with their interaction, and as dependent variable the spike count in the cue period. We found that only 2/15 neurons showed a significant interaction term (two-way ANOVA, p1 = 0.029, p2 = 0.046). This indicates that the two signals (strategy and switching) are in most of the cases (13/15 neurons) independent from each other.
Additionally, we investigated whether neurons recorded in the dorsolateral prefrontal cortex (PFdl), within the same dataset (Tsujimoto et al., 2010, 2011, 2012), were modulated by the switching signal in the cue period. The PFdl database consisted of 551 neurons recorded during the visually cued strategy task. We applied the same screening criteria described above (see Materials and Methods) and the 19% of these cells were classified as task-related neurons (n = 102). We performed a one-way ANOVA (p < 0.05) and a permutation test following the same procedure adopted to analyze PFo neurons activity (see Materials and Methods). We obtained a percentage of neurons modulated by switching signal equal to 8/102 (∼8%). Nevertheless, the observed proportion of switching neurons was not statistically different from shuffled data (p = 0.15).
Population analysis
To quantify the strength of the switching selectivity and its time course, we computed the auROC for the switching conditions for the population of 15 switching selective neurons. We also computed the auROC, for the same neuronal sample, for the strategy (stay/shift). Figure 5 illustrates the time course of the mean auROC for both the strategy switching (red line) and the strategy (gray line). As shown, the coding strength of strategy and switching is comparable: it increased during the cue period, reaching its peak at ∼300 ms after cue onset for both the signals. The mean auROC value for the switching strategy calculated in the cue period was equal to 0.649 (SEM = 0.003), and for the strategy was equal to 0.62 (SEM = 0.02).
Mean value of the auROC for the strategy switching (red line) and the strategy (gray line) conditions. The auROC was computed in a 200 ms time window stepped by 5 ms. The mean auROC was calculated averaging over the auROC values of the 15 switching neurons. The thin lines indicate the SEM.
Moreover, to investigate the strength of the switching signal and its dynamics at the level of the whole population (124 task-related neurons), we performed a neuron-dropping analysis using for classification a PSTH classification method (Foffani and Moxon, 2004). We considered the time period from 500 ms before the cue onset up to 500 ms after the cue offset. We used a time bin of 200 ms stepped by 50 ms. In Figure 6, we show the result of the neuron-dropping analysis. We found that, as the sample's size increased (y-axis), the accuracy of test in classifying the conditions increased (z-axis, color bar), reaching the maximum of performance during the presentation of the visual cue (x-axis). This result shows how the whole neural ensemble contributed to some degree to the decoding of the switching signal, as seen by the peak of decoding performance higher than the classification accuracy obtained by the mean of the single auROC of the 15 switching neurons (Fig. 5, red line). This population analysis supports and bolsters the previous result based on a single-cell analysis.
Neuron-dropping analysis applied to the whole population of task related neurons (124 neurons) to decode the switching strategy signal. The x-axis is the time aligned at the cue onset, the y-axis is the number of neurons constituting the neuronal sample used to perform the decoding (from 10 neurons up to 124 neurons), and the z-axis (color bar) shows the mean percentage of the correct classified trials, as nonswitch or switch, after 1000 repetitions. We computed the spike activity in a bin size of 200 ms stepped by 50 ms.
We also examined the switching selectivity in the error trials using the bootstrap analysis explained in Materials and Methods. We tested whether the observed population activity difference between the preferred and anti-preferred conditions was significantly different from that obtained with shuffled data for both correct and error trials. It is worth underlining that we assigned the preferred condition of the correct trials to the error trials, for each neuron. For instance, if a neuron preferred the switch condition in correct trials, then in error trials the switch condition was addressed as “preferred”, regardless of the activity on those trials. We restricted the analysis to 13 of the 15 neurons that had at least one error trial for each condition.
Figure 7 shows the mean activity difference (gray square) for correct (abscissa) and error (ordinate) trials between preferred and anti-preferred strategy switching conditions. As illustrated in Figure 7, the observed activity difference exceeded all 1000 points of shuffled data for both correct and error trials. As we defined the preferred condition in error trials as the same as for the correct trials, the negative value of the activity difference in the error trials indicates that, on average, there was a change of preference between the conditions associated to correct and error trials. This result shows that the strategy switching activity of the cells reflected whether the monkeys switched strategy or not even in case of mistake.
Mean activity difference between preferred and anti-preferred strategy switching conditions for the observed (gray square) and shuffled (black points) data. The error bars are the SEM. The x-axis and y-axis show the mean activity difference between preferred (pref) and anti-preferred (anti) conditions for correct and error trials, respectively. The shuffled-data distributions are shown on top and on the right for correct and error trials, respectively.
Discussion
In the present study, we investigated whether PFo shows strategy switching-related activity while monkeys performed a strategy task, switching between two strategies across consecutive trials (Tsujimoto et al., 2011, 2012). We found that PFo but not PFdl contains a significant portion of recorded neurons modulated by strategy switching during the cue period in which the strategy was instructed. This result extends the information encoded by PFo neurons from the strategy itself (stay or shift) (Tsujimoto et al., 2011), as in PFdl but not in the Frontal Pole (Tsujimoto et al., 2010, 2011) to the strategy switching.
We also examined the relationship between the strategy switching and strategy signal. We observed the highest proportion (23%) of neurons with strategy switching-related activity within the population of the strategy neurons pointing to a relationship between strategy coding and switching activity. This relationship suggests that, at least for a subpopulation of strategy cells, the integration with the switching signal might play a role in strategy generation.
Possible roles of the strategy switch-related signal in PFo
It is important to note that, in our paradigm, the strategy switching signal was not required for guiding the behavioral choice. Indeed, our strategy task did not require the subjects to take into consideration either the strategy switching or the sequence or history of strategy changes across trials for making the correct choice. We propose two possible roles of the switching signal in our task.
First, this signal could facilitate the strategy generation, helping to inhibit any automatic behavior such as the repetition of the past strategy, a function traditionally assigned to PFo (Roberts and Wallis, 2000), or a continuous strategy change. Second, the strategy switching signal can contribute to a general task-monitoring function. According to this hypothesis, the strategy switching-related activity in PFo might be involved in some monitoring process of the task strategy, although irrelevant to the task performance given the stability of the task contingencies. Monitoring signals of past events and previous behaviors are important and can promote changes that optimize the behavior in response to environment changes (Passingham and Wise, 2012; Tsujimoto and Postle, 2012; Genovesio and Ferraina, 2014; Genovesio et al., 2014). However, if this were the case, we would expect a monitoring of other important variables of the tasks during the cue period and not limited to the switching information. Instead, we have previously reported that PFo neurons are not modulated, in contrast to PFdl, by the previous target location although critical to the task performance (Tsujimoto et al., 2012) and furthermore PFo neurons lose their modulation for staying or shifting of the target location in a control delayed-response task with no strategy requirement.
In favor of a role of the switching signal useful for strategy generation is the timing of the switching modulation. The switching modulation occurred during the presentation of the strategy cue and dissipated soon after, suggesting that this signal could help the implementation of the strategy based on the cue information. Second, the error analysis showed that, during errors, the activity of the switching cells reflected whether the monkeys erroneously switched strategy or not, instead of whether the trial required or not a strategy switch.
Comparison with previous studies
In the present study we have shown that PFo activity was modulated by strategy switching, using an experimental paradigm in which the switching signal, error signals, and reward expectation were dissociated by task design. Strategy switching depended only on the repetition or change of the strategy from one trial to the next one and not on the behavioral outcome. This aspect of the task design allowed a clear dissociation between switching and error signals, in contrast to previous neurophysiological studies in which the switch followed the absence of reward (Mansouri et al., 2006; Kamigaki et al., 2009; Yamada et al., 2010; Sleezer et al., 2017). In addition, we also note that the switch-related activity could not be explained by differences in visual responses, because both switch and nonswitch trials occurred in equal proportions after the presentation of stay and shift cues. Because PFo is not coding neither the previous nor the current response during the cue period, we can also rule out that our switching modulation depends on some nonlinear combinations of strategy and response.
PFo has been considered critical for behavioral flexibility in neuropsychological studies in humans (Hornak et al., 2004), macaques (Izquierdo et al., 2004), and marmosets (Dias et al., 1996). Similarly, previous neurophysiological studies that investigated the switch-related activity in PFo (Chase et al., 2012; Sleezer et al., 2017) have suggested that this area plays a role in set reconfiguration, and these properties may play a role in guiding target selection (Sleezer et al., 2017). However, a recent lesion study that spared the fibers running near or through PFo, using excitotoxic lesions rather than aspiration, has failed to find deficits in tasks requiring a shift in the object reversal task (Rudebeck et al., 2013) whereas it affected the performance in a devaluation task. This negative result suggests the need for reconsideration of the effect of the earlier lesion studies in macaques because the deficits could depend on the lesion of fiber tracts rather than lesion of the PFo cortex. To reconcile our neurophysiological results and the recent negative results of the PFo lesion in macaques (Rudebeck et al., 2013), it could be assumed that PFo might contribute to behavioral flexibility by controlling behavioral switch but that its lesion can still be compensated by other prefrontal areas. We should also consider the possibility that the switching signal as described in our study does not contribute to reversal learning. Reversal learning tasks involve a specific type of switching that requires the inhibition of the previous association for the reversal of the reward contingencies (Dhawan et al., 2019) which is not required in other switching tasks (Stoet and Snyder, 2003; Avdagic et al., 2014). Furthermore, even the reversal tasks are not all alike as shown by La Camera et al. (2018) that found reversal deficits of concept-based rules but not of visual outcome association rules. Future studies with different paradigms should also address whether the switching signal as we described in PFo can contribute to a flexible behavior by promoting the use of new shortcuts instead of well learned strategies when the shortcuts represent a more efficient alternative (Watzek et al., 2018).
Previous studies have reported the switch-related signal in PFo neurons (Sleezer et al., 2016, 2017), but our experiment is different in that the switch is regarding the abstract information in contrast to the concrete information. Notably the current switch signal shows that the switch signal does not depend necessarily on the evaluation of the trial outcome, and rather it occurred independently from error signals.
The switch-related signal has been found in a wide variety of brain areas but with different paradigms that make the between-areas comparison hard. For example, such signal has been described using an analog of the Wisconsin card sorting test in the striatum (Sleezer et al., 2016) and in the parietal cortex (Kamigaki et al., 2009). Kamigaki et al. (2009), for example, found in the parietal cortex an increase of activity specific to the shifting trials and only for the shift to color and not for the shift to shape, while in our study it was more general and bidirectional. Sleezer et al. (2016) reported the switching-related signal for switching of more concrete visual stimuli rather than categories in the dorsal and ventral striatum. In contrast, no switching signal was reported by Mansouri et al. (2006) in dorsal prefrontal cortex, notwithstanding its role in the executive function in line with our results on PFdl in the cue period. Other neurophysiological studies have considered the switching signal for actions rather than for cognitive sets of rules considering the switching of action involving either an arm movement or a saccade. Shima and Tanji (1998) described the switching signal from one action to another as an effect of reward reduction for one specific action in the cingulate motor area, and Isoda and Hikosaka (2007) reported the switching signal in the presupplementary motor area in a switching action paradigm when switching from an automatic to a volitionally controlled action. We have to point out that, whereas a behavioral switch is required in the tasks discussed before, it is not necessarily present in all forms of flexible behaviors e.g., in contexts in which the flexibility emerges during cognitive control (Wallis et al., 2001b) or in innovative behavior and problem solving (Heubner and Fichtel, 2015) in which no behavioral switch is required. We propose that the switching signal in PFo can be important to facilitate a rule or strategy change and contributing to the inhibition of the tendency to implement the most recent strategy, or more generally helping to override any automatic behavior in the strategy implementation process.
Conclusions
In conclusion, we have found that PFo neurons were modulated by strategy switching and strategy (Tsujimoto et al., 2011, 2012), pointing to the importance of PFo in behavioral flexibility by generating a control over the strategy switching. Investigating the role of PFo in behavioral switching is important because it can help to understand psychiatric disorders such as the obsessive-compulsive disorder (Bannon et al., 2006; Gu et al., 2008; Menzies et al., 2008) in which behavioral flexibility is compromised, and to develop new therapies, e.g., PFo cathodal transcranial direct current stimulation that has been effectively used to reduce obsessive-compulsive behavior (Mondino et al., 2015).
Footnotes
This work was supported by the Division of Intramural Research of the National Institute of Mental Health–National Institutes of Health (Grant Z01MH-01092), Avvio alla Ricerca 2018 Tipo 1, Università La Sapienza (Grant AR118164313D0C16), and Ateneo 2019 Università La Sapienza. We thank Steven P. Wise and Andrew R. Mitz for support during all phases of this project and James Fellows and Ping-Yu Chen for technical support.
The authors declare no competing financial interests.
- Correspondence should be addressed to Satoshi Tsujimoto at satoshi.tsujimoto{at}nielsen.com or Aldo Genovesio at aldo.genovesio{at}uniroma1.it
