Abstract
When Pat Goldman-Rakic described the circuitry and function of primate prefrontal cortex in her influential 1987 monograph (Goldman-Rakic, 1987), she included only a few short paragraphs on the orbitofrontal cortex (OFC). That year, there were only nine papers published containing the term “orbitofrontal,” an average of less than one paper per month. Twenty years later, this rate has increased to 32 papers per month. This explosive growth is partly attributable to the remarkable similarities that exist in structure and function across species. These similarities suggest that OFC function can be usefully modeled in nonhuman and even nonprimate species. Here, we review some of these similarities.
What do we know about orbitofrontal cortex from studies of reversal learning?
Historically, OFC has been linked to inflexible or inappropriate responding. This deficit is evident in Dr. John Harlow's account of the most famous orbitofrontal patient, Phineas Gage (Harlow, 1868), and it is most often demonstrated experimentally in reversal learning tasks. Reversal learning occurs when an animal must alter established response tendencies when outcome contingencies change, typically in discrimination tasks (Fig. 1A). Thus, the animal is taught that responding to one cue produces reward, whereas executing the same response to another cue produces nonreward or punishment. After the animal learns to respond correctly, the experimenter switches the cue–outcome associations, and the animal must learn to change its behavior. Rats, cats, mice, marmosets, monkeys, and humans with damage to OFC are slower to acquire reversals (Mishkin, 1964; Teitelbaum, 1964; Butter, 1969; Dias et al., 1996; Bechara et al., 1997; Ferry et al., 2000; Chudasama and Robbins, 2003; Fellows and Farah, 2003; Schoenbaum et al., 2003; Hornak et al., 2004; Izquierdo et al., 2004; Bissonette et al., 2005), and neural activity related to the cue–outcome learning across reversals is evident in OFC of rats, monkeys, and humans (Thorpe et al., 1983; Schoenbaum et al., 1999; O'Doherty et al., 2003b; Wallis and Miller, 2003; Hampton et al., 2006) (Fig. 1B,C). Similar results are obtained despite the methodological differences in these studies. For example, lesions can be neurotoxic or by aspiration, punishment can be explicit or consist of a time-out, the task can be go, no-go, or symmetrically rewarded, and the outcome contingencies may be all-or-nothing or probabilistic.
Neural responses in both rat and human OFC during reversal learning signal expectations of a subsequent reward outcome. A, Illustration of generic discrimination reversal task in which the subject is presented with two stimuli and on each trial gets to choose one. One of the stimuli, if chosen, yields a reward (illustrated here as the cherries), whereas the other stimulus yields either an aversive outcome or simply the absence of reward (no cherries). During the acquisition phase, the subject learns to discriminate between the two stimuli, choosing the stimulus associated with reward and avoiding the nonrewarding stimulus. After subjects have reached a criterion (typically after choosing the rewarded stimulus on a number of consecutive trials), a reversal phase ensues, during which the previously rewarded stimulus now yields nonreward, and the previously unrewarded stimulus now yields a reward outcome. Reversals can occur multiple times during task performance. B, Plot of spiking activity recorded extracellularly from a neuron in rat OFC during learning and reversal of an odor discrimination problem. One odor predicted availability of sucrose at a nearby well, whereas a second odor predicted quinine. Data are shown during and after learning (precriterion vs postcriterion) and after reversal. Each raster display shows neural activity time locked to odor onset. Gray shading shows odor sampling, and green or red shading indicates response to the fluid well, which was followed by delivery of sucrose or quinine at the end of the shaded region. Average activity is summarized in the perievent time histogram below each raster. This neuron exhibits elevated activity to sucrose and also during the delay in anticipation of sucrose during learning. After learning, the neuron also becomes active during sampling of odor 1, the sucrose-predicting odor cue. After reversal, the activity of the neuron tracks the sucrose outcome, rapidly changing to fire in anticipation of sucrose on odor 2 trials and slightly later during becoming active during actual sampling of odor 2. Data are from Stalnaker et al. (2007). C, Statistical map of BOLD responses from a functional magnetic resonance imaging study of reversal learning in humans (left), depicting activity in medial orbitofrontal cortex (circled). This image is from an analysis to detect areas correlating with expected reward in the interval between cue presentation and outcome delivery. Activity in medial OFC and adjacent medial prefrontal cortex is approximately linear with respect to the expected value of the outcome, as indicated by the plot of activity in this region against expected value derived from a computational model. Other areas showing significant effects in this analysis include the amygdala and anterior hippocampus bilaterally. Color bars depict t-statistic level. Errors bar depict SEM. Data are from Hampton et al. (2006).
Thus OFC appears to be fundamentally critical to facilitating rapid reversal learning, but why? Although this deficit has often been attributed to OFC's role in inhibiting “prepotent” responding, typically animals are able to inhibit the same response before learning that they are unable to inhibit after reversal. Moreover, OFC lesions do not affect reversal of some naturalistic or innate response tendencies (Chudasama et al., 2007). This suggests that OFC is not critical for reversal learning because of a general role in response inhibition. Instead, there must be some deeper underlying function critical for guiding behavior when outcomes change. Insight into this function has come in the last several years through the use of reinforcer devaluation tasks.
What do we know about orbitofrontal cortex from studies of reinforcer devaluation?
Reinforcer devaluation refers to a procedure in which changes in a learned response are assessed after devaluation of the expected outcome. Typically, this is done with food by selectively satiating the animal on the outcome or by pairing it with illness (Holland and Straub, 1979). In contrast to reversal learning, in which the animal directly experiences pairing of the new outcome with the cues and responses, the effects of reinforcer devaluation do not involve a change in the actual outcome and can be observed under extinction conditions. Under these conditions, the animal must use preexisting information, updated with the new outcome value, to appropriately guide behavior. This complex feat, which requires recall of the original associations and integration with the new outcome value, is easily accomplished by normal animals, who respond significantly less for a devalued than a nondevalued outcome. In contrast, rats and monkeys with OFC lesions fail to alter learned responding after devaluation (Gallagher et al., 1999; Izquierdo et al., 2004). This is true even when OFC lesions are made after acquisition of the stimulus–outcome associations or even after outcome devaluation (Pickens et al., 2003, 2005), suggesting that OFC plays a critical role in evoking the original learning and integrating it with the new value of the outcome to guide responding. Consistent with this proposal, neurons in OFC modify their responses to food items and to predictive cues to reflect changes in their current incentive value (Critchley and Rolls, 1996; Tremblay and Schultz, 1999; Hikosaka and Watanabe, 2000; Gottfried et al., 2003; Scott et al., 2005). Thus, OFC is critical for signaling the current value of the expected outcome.
Questions for the next 20 years
Although the last 20 years have brought remarkable progress, this work has also uncovered interesting questions; here we highlight several.
How (When?) does OFC facilitate flexible behavior?
OFC is important for outcome-guided behavior (devaluation) and also for facilitating changes in that behavior in the face of unexpected outcomes (reversal). Are these functions dissociable, or do they derive from a common “precursor” function? The involvement of OFC in reversal learning may reflect its critical role in signaling expected outcomes, which may contribute to teaching signals early in reversal (Stalnaker et al., 2007). Alternatively, reversal learning may rely on rules regarding the occurrence rather than the value of reward per se (Murray and Izquierdo, 2007), suggesting different underlying mechanisms underlying OFC's role in the two tasks. Consistent with this idea, neurons in OFC clearly encode abstract rules that predict the occurrence of reward (Ramus and Eichenbaum, 2000; Wallis et al., 2001), and preliminary data suggest a possible dissociation of devaluation and reversal abilities within subregions of OFC (Kazama and Bachevalier, 2006). However, value signals in human orbital and medial prefrontal cortex are modulated by the rules or structure of reversal tasks (Hampton et al., 2006). Disentangling these functions may prove difficult.
How does OFC represent apples versus oranges … and footshock?
Research on OFC function is converging with the rapidly developing field of neuroeconomics. A central tenet of neuroeconomics is that the brain has a common currency in which to represent the value of goods, so that decisions between dissimilar items, like apples and oranges, can be made efficiently. This role has been assigned to OFC (Montague and Berns, 2002); OFC neurons are activated by rewards and cues that predict rewards in a way that reflects their incentive value, typically independent of the predictive cue or associated response (Tremblay and Schultz, 1999; Arana et al., 2003; O'Doherty et al., 2003b; Wallis and Miller, 2003; Roesch and Olson, 2005). Consistent with this proposal, OFC neurons in monkeys that are trained to perform a complex decision-making task for differently valued rewards seem to provide a pure, linear representation of the juices' value (Padoa-Schioppa and Assad, 2006). These data show that OFC can signal apparent economic value. However, a number of issues remain to be addressed. For example, how specific is this result to OFC and to this particular setting? Is a similar signal observed in other brain areas or when decisions must be made spontaneously, in a novel context or between novel goods? Or is such pure value encoding only observed in OFC in expert decision makers? Data from rats learning to make decisions suggest that economically relevant variables, in this case response direction as well as time to and size of reward, are represented independently within OFC (Roesch et al., 2006). Although this may be a species difference, it may also be that circumstance and familiarity are critical in determining how tightly encoding in OFC integrates disparate information bearing on value. It will also be critical to know how aversive outcomes are integrated into this common currency. OFC represents bad outcomes as well as good ones (O'Doherty et al., 2001; Gottfried et al., 2002; O'Doherty et al., 2003a; Morrison and Salzman, 2006).
Does OFC discriminate between pavlovian (stimulus-outcome) and instrumental (response-outcome) associations?
A role for OFC in signaling expected outcomes has been demonstrated thus far only in pavlovian settings. This includes the explicit pavlovian reinforcer devaluation task described above (Gallagher et al., 1999) as well as operant tasks built on object discrimination learning (Izquierdo et al., 2004), in which devaluation likely operates on pavlovian associations between the cues and the differently preferred food rewards. Whether OFC also plays a role in response-outcome or “goal-directed” instrumental learning is less clear. Recently, one of us has shown that blood oxygenation level-dependent (BOLD) activity in human OFC during the selection of actions leading to different outcomes is modulated as a result of reinforcer devaluation (Valentin et al., 2007). Similarly, two different laboratories have shown that neurons in rat OFC fire differentially for left and right responses, even when they lead to the same reward (Feierstein et al., 2006; Roesch et al., 2006). However, large lesions of rat OFC fail to disrupt changes in instrumental responding after reinforcer devaluation (Ostlund and Balleine, 2007). Several accounts may reconcile these findings. First, although information linking responses and outcomes may be present in OFC, it may not be essential for behavior in instrumental settings. Apparent response–outcome correlates may also reflect sensory attributes of different responses (e.g., space). Alternatively, portions of rat and primate “medial” OFC may be more closely related to medial prefrontal areas already implicated in guiding behavior based on action–outcome associations (Rushworth et al., 2007). Additional study will be needed to evaluate these proposals.
What is OFC anyway?
A common theme in many of the questions discussed above is how to define OFC: what are the critical anatomical features that describe an area as orbitofrontal, both within and across species, and are there specialized areas within OFC that mediate particular functions? It will be increasingly critical in future studies to be precise in defining subregions within OFC. For example, future work will likely uncover differences in OFC function that reflect evolutionary specializations between species. One region of particular interest in this regard may be the rostral portion of OFC in primates, which is characterized by a granular cell layer that is not evident in rats (Preuss, 1995). Another open question is the nature of the functional relationship between rostral parts of OFC and adjacent regions of frontopolar cortex implicated in exploratory decision making (Daw et al., 2006). In addition, future studies should address the way in which autonomic signals affect processing within OFC (Roberts, 2006). Determining how regions within OFC promote flexibility, represent potential positive and negative consequences of choices, and guide selection of the most advantageous choices will be a highlight of the coming 20 years. This work will help elucidate the neural underpinnings of adaptive behavior and may also shed light on unique features of the human character.
