Distinctive Roles for the Ventral Striatum and Ventral Prefrontal Cortex during Decision-Making

A problem well known to most cognitive neuroscientists, particularly those in brain imaging, is one of correlation. In designing a task to distinguish the function of a given brain region, how can we be sure that the activity we observe does not merely correlate with another, superior descriptor of this region's function? The question is particularly relevant in studies of economic decision-making, or “neuroeconomics,” in which several key computational metrics are closely related to one another but have been reported to be encoded in overlapping regions of the brain.

One such region is the ventral striatum. Knowing (and thus encoding) the expected value of a possible course of action is considered critical if it is to be successfully weighed against other available options. Both monkey single-unit data (Samejima et al., 2005) and human functional magnetic resonance imaging (fMRI) data (Kable and Glimcher, 2007) have pointed to the ventral striatum as one of several key regions for encoding this metric. However, this region also receives dopaminergic input from the ventral tegmental area (VTA) in the midbrain. The VTA signals “prediction errors,” differences between expected and received levels of reward, considered important by formal theories of learning (Schultz et al., 1997). These dopaminergic inputs are believed to underlie a prediction error signal frequently witnessed in the ventral striatum during human fMRI studies of learning and decision-making (O'Doherty et al., 2006). Often, in experimental paradigms demonstrating the role of the ventral striatum in signaling action values, trials containing high value options are inherently more rewarding, and so a positive prediction error might be expected. Thus the following question emerges: are these action value and prediction error signals distinct from one another, or might they really be one and the same thing?

This was the issue tackled in a recent article by Hare et al. (2008) in The Journal of Neuroscience. The authors designed a task in that the correlation between prediction errors and action values was carefully minimized, allowing them to distinguish brain activity relating to each. Subjects were given a series of decisions in which they chose whether to spend part of $5 “spending money” on an item of food. The authors used a prescan auction to obtain estimates of how much subjects were willing to pay for each item which they termed “goal values.” Each food item was displayed with a “price,” ranging from −3 to +$3, which the subjects could choose to spend on the item, and another value, spanning the same range, which the subjects would gain or lose regardless of their decision.

The overall combination of goal value and price (i.e., value of food gained minus money spent) was termed the “decision value.” Thus, if a bar of chocolate valued by the subject at $1 was presented, but the price of this bar was set at $3, the goal value would be $1 and the decision value −$2 ($1 − $3). The authors expected goal values and decision values, two related but dissociable features of action selection, to both be encoded to allow successful decision-making. The prediction error was modeled as the difference between the overall outcome of the current trial (the gain/loss, plus the decision value if the food item had been chosen) and the average outcome of recent trials (or the “expected value” of a trial). So, suppose that the trial with the chocolate described above was accompanied by a certain loss of $3, but that the expected value based on recent trials was +$1. The prediction error for that trial would be −$4 if the subject chose not to buy the chocolate (−$3 − +$1), and −$6 if the subject chose to buy the chocolate (−$3 − $2 − +$1). By manipulating the food item displayed, the price and the gain/loss independently from trial to trial, and by including some trials on which each of these components was absent, the authors greatly reduced the degree of correlation between goal values, decision values, and prediction errors.

Subjects' choices inside the scanner closely matched their behavior during the prescan auction. If the decision value was $0, subjects chose the food item on approximately one-half of the trials; above this value, they chose the item frequently, and below this value, they chose it infrequently [Hare et al. (2008), their Fig. 1c (http://www.jneurosci.org/cgi/content/full/28/22/5623/F1)]. This is important, because it demonstrates that the goal values (and decision values) used as regressors in analysis of the fMRI data accurately reflected the subjects' own valuation of the items on offer.

It was found that blood oxygenation level-dependent fMRI signal in the ventral striatum correlated significantly with prediction error [Hare et al. (2008), their Table 3 (http://www.jneurosci.org/cgi/content/full/28/22/5623/T3) and their Fig. 2f (http://www.jneurosci.org/cgi/content/full/28/22/5623/F2)] but not with goal value or decision value [Hare et al. (2008), bottom of their Fig. 3 (http://www.jneurosci.org/cgi/content/full/28/22/5623/F3)]. This is a surprising result, because the prediction error signal is traditionally thought of as being important for learning, but the experimental task does not require any learning across trials. Indeed, the authors emphasize that because only one of the subjects' decisions would be randomly selected at the end of the experiment as being “for real,” subjects could treat each decision as if it were the only one that counted. This would imply that even when performing tasks that allow trials to be treated independently, subjects (consciously or otherwise) keep a running track of how valuable each trial is likely to be. A positive prediction error is then witnessed when a particularly valuable trial is presented [Hare et al. (2008), their Fig. 2o (http://www.jneurosci.org/cgi/content/full/28/22/5623/F2)]. Such cross-trial learning bears striking similarities to the “anchoring effect” frequently described in economic research, in which the scale on which a previous judgment was made strongly influences categorically similar future judgements (Strack and Mussweiler, 1997). The authors suggest that this result might also explain why recent studies (Kable and Glimcher, 2007) have identified the ventral striatum as encoding the value of a chosen action: this is, in fact, a prediction error, signaling the onset of trials in which more valuable options are presented. This makes some sense because, theoretically, every event should be encoded in terms of its prediction error, not just events involving the delivery of reward.

The authors also identified other regions that appeared to encode goal values and decision values [Hare et al. (2008), their Table 1 (http://www.jneurosci.org/cgi/content/full/28/22/5623/T1) and their Table 2 (http://www.jneurosci.org/cgi/content/full/28/22/5623/T2)] but not prediction errors. The goal value signal was found on the medial wall and medial orbital sulcus of ventral prefrontal cortex, whereas the decision value signal was located in the more lateral orbital gyrus. Although very close to one another, anatomical tracer studies in macaques have suggested that these regions form components of distinct connectional networks (Price, 2007), which could underlie their roles in signaling distinct computational metrics during decision-making.

However, a technical but important feature of the fMRI analysis should be considered before these different components are attributed uniquely to their respective brain areas. Despite the authors' efforts to minimize the degree of correlation between the three regressors of interest, there nonetheless remained some shared variance between all three. If these partially correlated regressors were entered into the general linear model used to describe the data, then only the uncorrelated (or “orthogonal”) components of each regressor would be allowed to compete for variance in explaining the observed data. Instead, before the analysis, the authors orthogonalized the decision value regressor with respect to the goal value regressor, and the prediction error regressor with respect to both goal value and decision value regressors. This technique does not affect the results of the orthogonalized regressor but instead assigns any shared variance to the unorthogonalized regressor (Andrade et al., 1999). Thus, in their analysis, the goal value regressor is given explanatory power that might derive from decision value, and signal attributed to either value regressor might be explained by the prediction error. The extent to which this is true depends on the degree of correlation between regressors, but it is nevertheless unclear from this study whether each region identified in ventral prefrontal cortex really signals each metric uniquely. Most importantly, however, this caveat does not apply in the ventral striatum, in which no goal value or decision value signal is found despite being assigned more explanatory power than the prediction error.

The paper's most significant finding, that ventral striatal activity may be better explained as a prediction error signal than as reflecting the values of different options, is an important one, as numerous studies have implicated this region in encoding action values. This result will no doubt need further replication and clarification itself. One further implication would be that some region of the brain must keep track of how valuable each trial is expected to be, as well as how valuable the options are, to compute this prediction error; it would be interesting to see whether this region overlapped with the regions found as possibly encoding action values in ventral prefrontal cortex. Also, as the authors stress, this prediction error signal is only important because of its role in updating these values. Thus, future work must seek to understand the interactions of these different regions and how these interactions lead to the unitary fMRI signals found in isolated brain regions.

• Copyright © 2008 Society for Neuroscience 0270-6474/08/288658-02$15.00/0