A Role for Orbitofrontal Cortex in Reward-Modulated Conflict Adaptation

When faced with various competing response options, the ability to adjust our behavior is indispensable to achieve our goals. For example, when tourists are crossing a street in London, they experience conflict between the automatic tendency to look left, and the street signs alerting them to look right. To safely cross the street, they will need to resolve this conflict, for example by suppressing the automatic behavior. The processes behind this adaptive control of behavior have been collectively referred to as cognitive control (Botvinick et al., 2001), or the ability to overcome automatic or overlearned behavior in favor of more appropriate responses (Verguts and Notebaert, 2009). In an experimental setting, cognitive control is reflected in response adjustments made after experiencing response conflict. More specifically, performance is generally found to be enhanced on conflict trials (i.e., trials where two response options compete) that are preceded by conflict trials. This is called the conflict adaptation effect. A long-held research tradition has ascribed an essential role to the anterior cingulate cortex (ACC) in the pursuit of adaptive behaviors (Botvinick et al., 2001). Being sensitive to the expected value of control, the ACC is believed to send signals to the dorsolateral prefrontal cortex (dlPFC) to allocate cognitive control (Shenhav et al., 2013). Although this framework provides a powerful mechanistic explanation for cognitive control, recent reports have suggested that adaptive behavior might also crucially depend on other brain regions, such as the supplementary motor area (SMA; Bonini et al., 2014). Therefore, which brain regions apart from ACC and dlPFC are essential contributors to adaptive behavior remains an open question.

In a recent issue of The Journal of Neuroscience, Mansouri et al. (2014) address this question by examining the role of the monkey orbitofrontal cortex (OFC), superior-dorsal lateral PFC (sdlPFC), and posterior cingulate cortex (PCC) in conflict-induced control adjustments. In their study, monkeys performed a variant of the Wisconsin Card Sorting Test (WCST), using a matching-to-sample design. In this task, monkeys were presented with a sample (i.e., a symbol that could vary in shape and color), and subsequently had to select the correct target out of three symbols, either based on a match in color or a match in shape. Conflict level was manipulated by varying whether the two rules (i.e., shape and color) demanded the same or a different response. On half of the trials, one of three targets matched the sample in color and shape (low-conflict trials). On the other trials, one target matched the sample in color and another target matched the sample in shape (high-conflict trials). On these trials, monkeys had to resolve the competition between the two matching rules. The monkeys received a reward after every correct response. The relevant matching rule (i.e., color or shape) alternated between each block, and a block of trials ended without any notice whenever the monkey reached an 85% accuracy level. Successful performance on this task therefore required the constant integration of reward information to update the currently relevant rule when needed. Lesions in OFC, but not in sdlPFC or PCC, selectively influenced two indices of task performance. First, it impaired performance on high-conflict trials, resulting in a decrease of the number of rule switches (i.e., OFC-lesioned monkeys took longer to reach the 85% accuracy level). Second, the enhanced performance on high-conflict trials following a high-conflict trial (i.e., the conflict adaptation effect) completely vanished in the group with OFC lesions.

Aware of the well established association between OFC and reward processing (Walton et al., 2010; Rudebeck et al., 2013), Mansouri et al. (2014) report additional results obtained by single-cell recordings. Using the same WCST, they show that individual cells in OFC are sensitive to the level of conflict during both sample presentation and decision making. However, in the recording study only, a different set of samples was used for high- and low-conflict trials, which implies that the level of conflict was entirely predictable based on the sample identity. Since high-conflict trials are more error-prone and therefore less likely to be rewarded, the conflict-dependent activity in OFC cells during sample presentation and decision making could have been caused by a confound with reward expectancy, as discussed by Kepecs et al. (2008). Importantly, however, the OFC cell activity during high-conflict trials was independent of whether the rule had just changed (i.e., the early part of a block) or had remained unchanged for a longer time (i.e., the late part of a block). Since correct responses and hence reward expectations on high-conflict trials were lower for the early part of a block, the authors argue that their results are indicative of OFC being involved in conflict processing and not just in reward expectancies. Nonetheless, although the volatile nature of the task indeed induces high uncertainty after a rule change (i.e., the early part of a block), the possibility of an upcoming rule change might also enhance uncertainty in the late part of a block. That way, the reward expectancies may not have been different for early and late parts of a block. In other words, although OFC might play a crucial role in choice–reward associations that are often important for adaptive behavior, it remains an open question to what degree it plays a direct role in conflict processing. We suggest a function of OFC in reward-modulated conflict adaptation and argue that this neatly fits into existing accounts of conflict adaptation.

Conflict and reward are thought to be important modulators of learning. For example, conflict is suggested to trigger an arousal response that strengthens task-relevant associations through Hebbian learning. This would lead to improved control on the trial following conflict because of the increased activation of the relevant rule (Verguts and Notebaert, 2009). In the case of two competing rules, the system needs to rely on reward signals to decide which one is the relevant one. The OFC might play a vital role here, as it is found to be crucial for crediting a received reward to a particular choice (Walton et al., 2010; Rudebeck et al., 2013). In the study by Mansouri et al. (2014), it is indeed necessary to process reward to appraise performance. The OFC might act as a modulator here and send facilitative or inhibitive signals to brain regions that are evaluating task performance and implementing enhanced cognitive control. Hence, rather than playing a central role in cognitive control, the OFC might facilitate reward-instigated learning through its interaction with conflict-related areas, such as ACC and dlPFC (Botvinick et al., 2001). The supposed interactive effect of reward and conflict on adaptation is supported by the fact that when performance-contingent rewards are delivered, the absence of a reward results in decreased conflict adaptation (Braem et al., 2012). This suggests that if reward is offered on some but not all trials, conflict adaptation is impaired after trials where the reward signal is missing or no longer appraised. Lesions to the OFC might degrade reward appraisal and thereby indirectly hinder the conflict-induced strengthening of the relevant rule, leading to impaired conflict adaptation. Indeed, in the study by Mansouri et al. (2014), OFC-lesioned animals were slower to learn the currently relevant rule, as evidenced by the fact that they took more time to reach the 85% accuracy level. Moreover, neuronal responses to reward in OFC were modulated by the level of conflict experienced in the course of achieving the reward and this may reflect the interaction between conflict- and reward-induced learning. That is, conflict adaptation may be optimal when conflict-induced learning is reinforced by reward. This suggests that cognitive control requires a delicate interplay between areas encoding conflict (e.g., ACC), rule maintenance (e.g., dlPFC), and choice-reward associations (e.g., OFC).

In conclusion, the necessity of having an intact OFC for adaptive behavior to occur might have been caused by the specific task set-up in the study of Mansouri et al. (2014). A straightforward way to examine the validity of our interpretation would be to assess whether the detrimental effect of OFC lesions on cognitive control is indeed limited to situations in which reward processing is indispensable for performing the task. In standard conflict paradigms, the same task set is used throughout the experiment (e.g., in the classical Stroop task, the rule is to ignore word meaning and respond to ink color), making reward processing (and OFC involvement) superfluous to assess accuracy of performance. Given that animal studies typically make use of probabilistic tasks on which rewards are offered, this question remains to be answered.