We examined the activity of individual cells in the primate anterior cingulate cortex during an economic choice task. In the experiments, monkeys chose between different juices offered in variables amounts and subjective values were inferred from the animals' choices. We analyzed neuronal firing rates in relation to a large number of behaviorally relevant variables. We report three main results. First, there were robust differences between the dorsal bank (ACCd) and the ventral bank (ACCv) of the cingulate sulcus. Specifically, neurons in ACCd but not in ACCv were modulated by the movement direction. Furthermore, neurons in ACCd were most active before movement initiation, whereas neurons in ACCv were most active after juice delivery. Second, neurons in both areas encoded the identity and the subjective value of the juice chosen by the animal. In contrast, neither region encoded the value of individual offers. Third, the population of value-encoding neurons in both ACCd and ACCv underwent range adaptation. With respect to economic choice, it is interesting to compare these areas with the orbitofrontal cortex (OFC), previously examined. While neurons in OFC encoded both pre-decision and post-decision variables, neurons in ACCd and ACCv only encoded post-decision variables. Moreover, the encoding of the choice outcome (chosen value and chosen juice) in ACCd and ACCv trailed that found in OFC. These observations indicate that economic decisions (i.e., value comparisons) take place upstream of ACCd and ACCv. The coexistence of choice outcome and movement signals in ACCd suggests that this area constitutes a gateway through which the choice system informs motor systems.
During economic choice, subjects assign values to the available options; a decision is then made by comparing values. Substantial work in recent years examined these mental processes at the neural level. In particular, it was found that neurons in the primate orbitofrontal cortex (OFC) encode the subjective values of offered and chosen goods, that subjective values correlate with the blood oxygenation signal in the human OFC and ventromedial prefrontal cortex, and that lesions to these brain areas selectively disrupt choice behavior. This collective evidence led us and others to propose that economic choices might be based on values computed in these regions (Kable and Glimcher, 2009; Padoa-Schioppa, 2011). However, important questions remain open. For example, the mechanisms through which values are compared to make a decision are poorly understood. Furthermore, it remains unclear whether and how other prefrontal regions contribute to valuation and decision making. Most prominent in this respect is the anterior cingulate cortex (ACC)—a region often proposed to have a key role in various types of decision making (Lee et al., 2007; Rushworth et al., 2007; Rushworth and Behrens, 2008; Rangel and Hare, 2010; Wallis and Kennerley, 2010).
While the encoding of subjective value in the ACC has not yet been examined thoroughly, several lines of evidence suggest that neurons in this region might represent value. In early work, Niki and Watanabe (1979) found cells in the dorsal bank of the ACC that responded to the delivery of juice reward, an observation confirmed in subsequent studies (Ito et al., 2003; Matsumoto et al., 2003; Williams et al., 2004; Seo and Lee, 2007). More recently, Amiez et al. (2006) varied the probability and the quantity of juice. They found that ACC responses covary with the expected juice quantity. Other studies found that the neuronal activity in this region is modulated by the effort exerted to obtain a reward (Kennerley and Wallis, 2009a; Kennerley et al., 2009). Similarly, Hillman and Bilkey (2010) found that neurons in the rodent ACC are modulated both by food quantity and physical effort. Arguably, these studies did not provide an accurate measure of subjective value because they did not include a trade-off between qualitatively different goods. However, all the results are consistent with the hypothesis that a representation of value exists in the ACC.
In the OFC, the encoding of value presents three key traits. First, different neurons encode different choice-related variables—Offer Value cells encode the value of one of the two offers, Chosen Value cells encode the value of the chosen offer, and Chosen Juice cells (or Taste cells) encode the binary result of the choice process. Second, all these variables are encoded independently of the visuomotor contingencies of choice (in goods space). Third, the encoding of value is range-adapting—the same range of neuronal firing rates describes different ranges of values in different behavioral conditions. In this perspective, the current understanding of how values might be encoded in the ACC is rather superficial. At least three issues remain open. First, it is not clear whether ACC neurons encode pre-decision variables (e.g., Offer Value) or post-decision variables (e.g., Chosen Value, Chosen Juice). Second, it is not clear whether and how this representation of value depends on the visuomotor contingencies of choice. Indeed, some studies found that roughly one-half of ACC neurons are spatially selective (Amiez et al., 2006; Kennerley et al., 2009; Hayden and Platt, 2010), while other studies found that spatial selectivity in this area is modest at best (Ito et al., 2003; Nakamura et al., 2005; Seo and Lee, 2007). Third, the adaptability of this representation remains to be examined (but see Sallet et al., 2007; Kennerley et al., 2011). The current study was conducted to address these important questions.
Materials and Methods
The discrepancy in the literature as for whether neurons in ACC are spatially selective may be partly due to the fact that experiments in which monkeys respond with arm movements (as opposed to eye movements) generally confound spatial selectivity with exerted effort. Indeed, in a study that used both response modalities, “spatial selectivity” was more prevalent when monkeys responded with arm movements compared with eye movements (Kennerley et al., 2009). Another possible explanation is that different studies might have recorded from different regions of the cingulate sulcus. Indeed, different papers equally label as “ACC” brain regions located as far as 15 mm from one another, and on either bank of the cingulate sulcus (Koyama et al., 2001; Shidara and Richmond, 2002; Matsumoto et al., 2007). For these reasons, we used an economic choice task in which the actions associated with the offers bear the same cost (isometric saccade to the left and to the right). Furthermore, we explored extensively both banks of the cingulate sulcus.
Economic choice task, surgery, and recordings.
All the procedures for behavioral control, neuronal recording, and data analysis were similar to those used in previous studies of OFC (Padoa-Schioppa and Assad, 2006, 2008; Padoa-Schioppa, 2009). Two rhesus monkeys (L, female, 6.5 kg; V, male, 8.5 kg) participated in the experiments. The animals sat in an electrically insulated enclosure, their head was restrained, and the eye position was monitored with an eye-coil system (Riverbend Instruments; monkey L) or with an infrared video camera (Eyelink; SR Research; monkey V). In each session, the monkey chose between two juices labeled A and B, with A preferred. In each trial, the monkey initially fixated a small point at the center of a computer monitor (see Fig. 1a). After 1.5 s, two sets of squares (offers) appeared at the two sides of the fixation point. For each offer, the color of the squares indicated the juice type and the number of squares represented the juice quantity. Thus, in the trial depicted in Figure 1a, the monkey chose between 1 drop of grape juice and 4 drops of apple juice. The monkey had to maintain center fixation for a randomly variable delay (1–2 s), after which the fixation point was extinguished and two saccade targets appeared by the offers (radial distance from center fixation: 7° of visual angle). The animal indicated its choice with a saccade and had to maintain peripheral fixation for 0.75 s before receiving the chosen juice.
The quantities of the two offers and their spatial configuration varied pseudorandomly across trials. Across sessions, we used a variety of different juices, resulting in many possible juice pairs. Typically, the behavior of the animal presented a trade-off: if the less preferred juice was offered in sufficiently high quantity, the monkey would choose it. The relative value of the two juices was thus inferred from the indifference point by fitting a normal sigmoid. This measure, which reflects the subjective nature of valuation, was used to define several value-related variables tested in the analysis of neuronal activity (see Table 3). Thus, each cell was analyzed in relation to the choice pattern recorded in the same session.
Under general anesthesia, we implanted a head-restraining device and an oval recording chamber on the skull. The chamber (main axes, 50 × 30 mm) was centered on stereotaxic coordinates (A30, L0), with the longer axis parallel to a coronal plane. Recordings were performed on the two banks of the cingulate sulcus and extended 12 mm in the anterior–posterior direction, from A23 to A35 (with the corpus callosum extending anteriorly to A31 and A30 in monkeys L and V, respectively). Tungsten electrodes (125 μm diameter; Frederick Haer) were advanced with a custom-made system driven remotely. We typically used four electrodes each day. Usually, we advanced the electrodes by pairs (one motor for two electrodes), with the two electrodes placed at 1 mm from each other. Electric signals were amplified, filtered (high-pass cutoff, 400 Hz; low-pass cutoff, 6 kHz) and recorded (Power 1401; Cambridge Electronic Design). Action potentials were detected on-line, and waveforms (25 kHz) were saved to disk for off-line clustering (Spike 2; Cambridge Electronic Design). Only cells that appeared well isolated and stable throughout the session were included in the analysis. Thus, data sets included 1044 cells from the right hemisphere of monkey L (760 from ACCd; 284 from ACCv) and 860 cells from the left hemisphere of monkey V (365 from ACCd; 495 from ACCv).
All experimental procedures conformed to the NIH Guide for the Care and Use of Laboratory Animals and with the regulations at Washington University School of Medicine and at Harvard Medical School.
Neuronal activity: preliminary analysis.
All the analyses were performed in MATLAB (version R2010b; MathWorks). Behavioral choice patterns were analyzed as previously described (Padoa-Schioppa and Assad, 2006, 2008). Briefly, for every session, the percentage of B choices was plotted as a function of log(qB/qA), where qA and qB are the offered quantities of juices A and B, respectively. The percentage of B choices was then fitted with a normal cumulative distribution function (normal sigmoid). The relative value of the two juices was inferred from the flex point of the sigmoid. Neuronal data were analyzed in relation to the relative value measured in the same session.
Neuronal activity was analyzed in eight time windows: Preoffer (from 0.5 s before the offer to the offer), Postoffer (0–0.5 s after the offer), Late Delay (0.5–1.0 s after the offer), Pre-go (0–0.5 s before the “go”), Reaction Time (from “go” to saccade), Prejuice (0–0.5 s before the juice), Postjuice (0–0.5 s after the juice), and Postjuice2 (0.5–1.0 s after the juice). These are the same time windows defined in previous studies with the addition of the Postjuice2 time window, which was particularly relevant for ACCv. An “offer type” was defined by two quantities of juice offered to the monkey (e.g., [1A:3B]), independently of both the spatial configuration and the choice of the animal. A “trial type” was defined by an offer type and a choice (e.g., [1A:3B, 1A]), independently of the spatial configuration. A “trial typeLR” was defined by a trial type and the spatial configuration (e.g., [1A:3B, 1A, R], where the last “R” indicates that the chosen offer was presented on the right). A neuronal response was defined as the activity of a neuron in one time window as a function of either the trial type or the trial typeLR, as specified in the text.
Our analysis proceeded in steps. First, we submitted our data sets to a series of ANOVAs, in which we always imposed the significance threshold p < 0.001. We used this relatively conservative criterion to conform with the methods used in our previous papers and because of the large number of responses analyzed (1904 cells × 8 time windows). In essence, these analyses indicated that neurons in ACCd can be modulated by the trial type and/or by the movement direction, independently. In contrast, neurons in ACCv can be modulated by the trial type, but they are independent of the visuomotor contingencies of choice. Moreover, neurons in the two areas tend to respond in different time windows (see Results) (Tables 1, 2). Thus, in all subsequent analyses, we examined separately the data sets from ACCd and ACCv. For both data sets, “task-related” responses were identified with a one-way ANOVA (factor: trial typeLR). Only responses that passed this criterion were included in subsequent analyses.
Second, we defined a large number of variables that neurons in ACCd and ACCv might potentially encode (Table 3). Each neuronal response was fitted against each variable. Since responses can generally depend on the movement direction, the fit was done using an analysis of covariance (ANCOVA) using the variable as a predictor and grouping data by the movement direction. In a preliminary analysis, we examined the full ANCOVA model, including the interaction term. However, we observed that the interaction term was rarely significant (which confirmed the results of the two-way ANOVAs) (see Results). We thus report here only the results obtained using the “parallel” model. This model assumes that the encoding of the factor (variable) and the group (movement direction) are statistically independent. A variable was said to “explain” the response if the factor in the ANCOVA was significant (p < 0.05). For each variable, the ANCOVA also provided an R2. This was the total R2, computed including both the factor and the group. For variables that did not explain the response, we arbitrarily set R2 = 0. These criteria were used for all variables except the Mov Dir Only, for which we proceeded as follows. We computed the “horizontal lines” model of the ANCOVA, which assumes that firing rates only depend on the group (movement direction). The variable Mov Dir Only was thus said to explain the response if the group was significant in this analysis (p < 0.05). From this model, we also obtained the R2, which was set equal to 0 if the group was not significant. Importantly, the R2 thus obtained for the variable Mov Dir Only was always lower than that obtained for any other variable that explained the response. Hence, for any given response, Mov Dir Only provided the best explanation (highest R2) only if it provided the sole explanation.
The overall rationale of this approach (already used in previous studies of OFC) was as follows. On the one hand, we imposed a rather conservative criterion when we admitted individual responses in the analysis (ANOVA, p < 0.001), to weed out noise from the data set. On the other hand, we set a more liberal criterion when we identified the encoding of any given variable (linear fit, p < 0.05), to maintain high sensitivity. Responses from ACCd presented one additional issue. Consider a directionally selective response. In principle, the response could pass the one-way ANOVA criterion largely because of its spatial selectivity (as opposed to a dependence on the trial type). Yet, following the ANOVA, the encoding of any particular variable would be established with a relatively liberal threshold. As a consequence, this procedure might occasionally assess that a variable explains a response even though the dependence of the response on the trial type is weak. To obviate this issue, we imposed an additional criterion: any variable (except Mov Dir Only) could explain a response only if the factor offer type in the three-way ANOVA was significant at the liberal threshold of p < 0.05.
Based on this analysis, we identified for each area and for each neuronal response the variable that provided the best fit (highest R2). We then computed the number of responses best explained by each variable, separately for each time window. Finally, for each area, we identified the variables that best accounted for the neuronal population, as follows.
Variable selection analysis.
In general, although one can define many variables potentially relevant to a particular experiment, a small subset of them might suffice to describe a given data set. To identify the variables that provided the best account for our data, we used two methods of variable selection—stepwise and best subset—originally designed for multilinear regressions in presence of multicollinearity (Dunn and Clark, 1987; Glantz and Slinker, 2001) and previously adapted to analyze large neuronal data sets (Padoa-Schioppa and Assad, 2006). The procedures used here are essentially identical to those described previously.
The stepwise method—an iterative procedure—was applied as follows. At the first step, we selected the variable that provided the highest number of best fits within any time window. We then removed from the data set all the responses explained by this variable (across time windows). At the second step, we repeated the procedure with the residual data set. We defined the “marginal explanatory power” of a variable X as the percentage of responses explained by X and not explained by any other selected variable. At each step, we imposed that all selected variables have a marginal explanatory power of at least 5% (if necessary, we dropped previously selected variables that failed to meet the criterion once new variables were added; in practice, this situation never occurred for the present data sets). We thus continued the procedure until when any additional variable failed to meet the 5% criterion.
The stepwise method had the advantage of analyzing different time windows separately but in parallel (see Results). However, this method did not guarantee optimality because its results were path dependent. In contrast, the best subset method—an exhaustive procedure—identified the subset of variables that provided the highest explanatory power. For this method, we pooled responses from different time windows. For each possible subset of d variables, we computed the number of responses explained in the data set, and we identified the subset that explained the highest number of responses. We repeated this procedure for d = 1, 2, 3, etc. In general, if n(d) is the number of responses explained by d variables, the number of variables necessary to account for the data set (d*) can be identified either by an elbow in the function n(d) or by a threshold criterion (e.g., 90% of responses explained). In practice, for both of current data sets, there appeared to be a natural breaking point (see Results).
One question left open by both these methods is whether the explanatory power of the selected variables is significantly higher than that of other possible variables—an issue referred to as possible “overfitting.” To address it, we performed a post hoc analysis in which we compared the marginal explanatory power of each selected variable with that of discarded but highly correlated variables. The procedure has been described previously (Padoa-Schioppa and Assad, 2006). In a nutshell, we labeled X a selected variable (e.g., Chosen Value) and Y a discarded variable that was highly correlated with the selected variable and that would thus represent a “challenging” alternative (e.g., Total Value). First, we quantified the marginal explanatory power of the selected variable (nX) as the number of responses explained by that variable, not explained by the challenging variable, and not explained by any other variable in the best subset. Second, we quantified the marginal explanatory power of the challenging variable (nY) as the number of responses explained by that variable, not explained by the selected variable, and not explained by any other variable in the best subset. The best subset method ensures that nX ≥ nY. To establish the statistical significance of this inequality, we ran a binomial test. This analysis was repeated for each selected variable and for each challenging variable with a correlation of ρ ≥ 0.8 (Tables 4, 5).
Notably, the variable selection analysis described above was defined on spatially dependent responses, where each data point represents one trial typeLR. This procedure is most appropriate for ACCd, where neurons were indeed spatially selective. However, for data from ACCv, it is actually preferable to perform the analysis on nonspatial responses, where each data point represents one trial type. Indeed, because more trials contribute to each data point, the results obtained with the nonspatial procedure (previously used for OFC) are generally more robust. For clarity of exposure, we detail in the following the results of the variable selection analysis obtained for both brain areas using the same spatial procedure. However, we also report the main results obtained for ACCv using nonspatial responses. In all subsequent analyses of neurons from ACCv, we pooled trials for different movement directions.
Subjective value versus physical property: analysis of U-shaped responses.
The variable selection analysis indicated that many neurons in ACCd and ACCv encoded the Chosen Value. In a further analysis, we examined more specifically whether these “U-shaped” responses indeed reflected the subjective nature of value, as opposed to encoding any physical property of the juice. For this analysis, we separated trials in which the animal chose juice A or juice B. For each group of trials, we fitted the neuronal firing rate against the amount of juice chosen (and received) by the animal. For ACCd cells, this fit was performed with an ANCOVA (parallel lines); for ACCv, this fit was performed with a linear regression. In either case, the fit provided two slopes α and β for juices A and B, respectively. If both slopes differed significantly from zero (95% confidence interval) and had the same sign, the response was said to be “U-shaped.”
If U-shaped responses indeed encode the Chosen Value, the slope ratio α/β should provide a neuronal measure for the relative value of the two juices. This measure should thus reflect the subjective nature of value and should be indistinguishable from the behavioral measure of relative value obtained from the choice pattern (indifference point). To test this prediction, we performed an ANCOVA using the neuronal measure as a dependent variable, the behavioral measure as a predictor, and grouping data by the juice pair. (Data recorded with different juice pairs were grouped separately because relative values generally depend on the juice pair.) We used the full ANCOVA model and we included only juice pairs for which we had at least 20 U-shaped responses (6 pairs for ACCd; 7 pairs for ACCv; see Fig. 8).
Analysis of range adaptation.
The analysis of range adaptation closely followed the methods previously used for OFC (Padoa-Schioppa, 2009). In the following, we will indicate with φ the neuronal firing rate and with V the Chosen Value. The linear fit performed for the variable selection analysis provided for each response the intercept c0and the slope c1 such that as follows: (For responses from ACCd, the intercept c0 depends on the movement direction.) For each response, we defined the value range ΔVas the range of values chosen by the animal within the session and the minimum value V0 as the lowest value chosen within the session. In general, Equation 1 can be rewritten as follows: where φ0 = c0 + c1V0 is the baseline activity and Δφ = c1 * ΔV is the activity range. If the encoding of value is linear, the hypothesis that neurons adapt to the value range amounts to the hypothesis that neuronal parameters φ0 and Δφ do not depend on the value range ΔV.
Conventionally, we always expressed the variable Chosen Value in units of juice B (uB). Each neuron in our data set was recorded in one session, which typically included 200–400 trials. Within each session, the Chosen Value (V) varied from trial to trial, within a range (ΔV). This range varied from session to session depending both on the offers and the animal's choices. Across sessions, the range of chosen values varied roughly twofold between 4 and 8 uB. To examine how the encoding of Chosen Value depended on the value range, we pooled responses from different time windows and we rectified responses with negative slopes. For each area, we then divided the neuronal population of responses in two groups, depending on whether the neuron was recorded in a session such that ΔV≤ 6 uB or ΔV> 6 uB. Finally, we compared the two groups of responses in several measures, including the distribution of regression slopes (c1), the activity range (Δφ), and the average normalized response.
To compute the average normalized response (see Figs. 9e, 10e), we subtracted the baseline activity φ0 corresponding to V = V0. We then divided the range of chosen values (x-axis) in 2 uB bins. For each bin and for each group of responses, we computed the average normalized firing rate (y-axis) and Chosen Value (x-axis) averaging across all relevant trial types and all responses. A similar procedure was used to compute averages when we examined the regression slope in relation to the value range (see Figs. 9c, 10c, color symbols).
Finally, we examined neuronal adaptation on the timescale of individual trials. In this analysis, we pooled trials for different movement directions, for both areas.
We recorded the activity of 1904 neurons from two animals performing economic choices (Fig. 1a). Our recordings extended 12 mm in the anterior–posterior direction and covered both banks of the cingulate sulcus (Fig. 1c,d). A preliminary inspection of individuals cells revealed clear differences between the two banks. Neurons in the dorsal bank and fundus (ACCd, 1125 cells) were often spatially selective and typically modulated during the delay. In contrast, neurons in the ventral bank (ACCv, 779 cells) were not spatially selective and most frequently modulated at the time of juice delivery. For all the population analyses, we thus examined separately neurons recorded in ACCd and neurons recorded in ACCv.
Identification of task-related responses
A first set of analyses were conducted to identify task-related neuronal responses, which would be subject to further examination. First, we submitted each neuron in each time window to a three-way ANOVA with factors (offer type by offer position by movement direction). In this analysis, the factor “offer position” captured the spatial left/right configuration of the offers on the monitor, whereas “movement direction” captured the left/right direction of the eye movement. Our results for ACCd and ACCv are summarized in Tables 1 and 2, respectively. In essence, neurons modulated by the offer type were present in both regions. In contrast, neurons modulated by the spatial configuration of the offers were very infrequent in either region (<2% of cells, pooling time windows). Finally, neurons modulated by the direction of the eye movement were fairly frequent in ACCd (20% cells, pooling time windows) but rather infrequent in ACCv (5% of cells, pooling time windows).
Starting with data from ACCd, we submitted each neuron to a two-way ANOVA with factors (trial type by movement direction) including the interaction term (results not included in Table 1). This analysis confirmed that both main factors modulated the activity of neurons in this area: pooling time windows, cells modulated by factors trial type and movement direction were 38 and 20%, respectively. In contrast, cells presenting a significant interaction term were very infrequent (<3% of cells, pooling time windows). In other words, the good-related modulation (captured by the factor trial type) and the action-related modulation (captured by the factor movement direction) were essentially independent from each other for any given cell. We thus repeated the two-way ANOVA including only the main factors, and we quantified how many neurons in ACCd were modulated by both factors (Table 1, sixth column). We found that the size of this subpopulation was very close from that expected by chance if the two modulations were completely independent (6% of cells modulated by both factors vs 7% expected by chance). In other words, the good-related modulation and the action-related modulation were independent across the neuronal population. For data from ACCv, the two-way ANOVAs confirmed that the neuronal activity could be modulated by the trial type, but was independent of the visuomotor contingencies of choice.
Finally, we identified for each area task-related responses using a one-way ANOVA (factor “trial typeLR”). Across time windows, task-related responses were 845 for ACCd and 359 for ACCv. Pooling time windows, neurons that passed the ANOVA criterion in at least one time windows were 406 of 1125 (36%) in ACCd and 194 of 779 (25%) in ACCv. These preliminary analyses also revealed that the two brain regions differed for the activation timing. Indeed, neurons in ACCd were modulated most prominently during the delay (time windows Late Delay and Pre-go). In contrast, in ACCv the modulation was most prominent immediately after juice delivery (time windows Postjuice and Postjuice2). This difference provided further evidence that ACCd and ACCv are physiologically distinct areas.
Choice-related variables encoded in ACCd
What choice-related variables are encoded in ACCd? A qualitative inspection revealed that neurons in ACCd often presented U-shaped responses, suggesting that they encoded the variable Chosen Value. As revealed by the ANOVAs (Table 1), many neurons were also modulated by the movement direction. However, these two factors appeared to be largely independent, in the sense that some neurons encoded only the Chosen Value (Fig. 2a,b), other neurons encoded both the Chosen Value and the movement direction (Fig. 2c), and yet other neurons encoded only the movement direction (variable Mov Dir Only; Fig. 2d). In addition, some neurons presented a binary response seemingly encoding the Chosen Juice (Fig. 2e). Of course, Chosen Value and Chosen Juice are only two of the many variables potentially encoded by this population. To identify the variables that best explained our data set, we adopted the same basic approach used in a previous study (Padoa-Schioppa and Assad, 2006).
Casting a wide net, we defined 24 variables that neurons in ACCd might potentially encode (Table 3). Specifically, we defined several good-based value-related variables (Offer Value, Chosen Value, Other Value, etc.), several quantity-related variables (Max #, Chosen #, Total #, etc.), a binary variable capturing the outcome of the choice process (Chosen Juice), and four spatial value-related variables (Offer Value L, Offer Value R, Chosen Value L, Chosen Value R). We also defined the variable Mov Dir Only, a purely spatial variable that captured responses modulated only by the movement direction (see Materials and Methods).
For each response and for each variable, we ran an ANCOVA, using the variable as a predictor and grouping data by the movement direction. A given variable was said to “explain” the neuronal response if the corresponding factor in the ANCOVA was significant (p < 0.05). As previously noted (Padoa-Schioppa and Assad, 2006), different variables included in this analysis could be highly correlated with one another (e.g., Chosen Value and Total Value were highly correlated in our experiments). Hence, any given response might be explained by more than one variable. In the light of this consideration, we generated two qualitative descriptions of our neuronal data set. Figure 3a illustrates for each time window the number of responses explained by each variable. Notably, any given response may appear in more than one bin in this plot. Conversely, Figure 3b illustrates for each time window and for each variable the number of responses best explained by that variable (highest R2). In this plot, each response appears in at most one bin. A qualitative inspection of Figure 3b revealed that the best explanation was often provided by Chosen Value. However, a sizable number of responses are best explained by Chosen Juice and/or by Mov Dir Only.
Overall, 804 of 845 (85%) task-related responses were explained by at least 1 of the 24 variables defined here. For a quantitative assessment of the variables that best explained the population, we used the stepwise selection method. As illustrated in Figure 4, a and b, the first three iterations of this procedure selected variables Chosen Value, Mov Dir Only, and Chosen Juice. Collectively, these three variables explained 763 responses. All variables selected in subsequent iterations provided a marginal explanatory power of <5%. The best subset method confirmed this result (Fig. 4c,d). As a single variable, Chosen Value explained more responses than any other variable. As a pair of variables, Chosen Value and Mov Dir Only explained more responses that any other pair of variables. Finally, the three variables Chosen Value, Mov Dir Only, and Chosen Juice explained more responses than any other subset of three variables. Again, these three variables collectively explained 763 responses, corresponding to 95% of the responses explained by the whole 24 variables, and to 90% of responses that passed the ANOVA criterion.
In summary, the explanatory power of variables Chosen Value, Chosen Juice, and Mov Dir Only was higher than that of any other subset of three variables. To examine whether this statement was true in a statistical sense, we ran a post hoc analysis in which the marginal explanatory power of each selected variable was tested against that of a discarded and “challenging” variable. As illustrated in Table 4, we found that the explanatory power of each selected variable was significantly higher than that of all challenging alternatives (all p < 0.05).
In conclusion, neurons in the ACCd appear to encode variables Chosen Value, Chosen Juice, and Mov Dir Only. On the basis of this result, we classified task-related responses from this area as encoding one of these three variables. Responses explained by more than one variable were assigned to the variable with highest R2. Responses not explained by any of these variables were labeled “unexplained.” In general, the encoded variable provided a good account of the responses [across variables, mean(R2) > 0.5; Fig. 4e].
Choice-related variables encoded in ACCv
A qualitative inspection revealed that neurons in ACCv often presented U-shaped responses, suggesting that they encoded the variable Chosen Value (Fig. 5a–c). Other cells presented a binary response suggesting that they encoded the Chosen Juice (Fig. 5d). These latter responses were most frequent in late time windows, after juice delivery. For a population analysis, we examined the same variables examined in the analysis of ACCd. For each response, we performed an ANCOVA on each variable (group: movement direction). We generated two descriptions of this data set. Figure 6a illustrates for each time window the number of responses explained by each variable. Figure 6b illustrates for each time window and for each variable the number of responses best explained by that variable (highest R2). A qualitative inspection reveals that the best explanation was often provided by the variable Chosen Value. However, especially in late time windows, a sizable number of responses were best explained by the variable Chosen Juice.
Overall, the 24 variables explained 337 of 359 (94%) responses that passed the ANOVA criterion. For a quantitative assessment of the variables that best explain the population, we first used the stepwise selection method. As illustrated in Figure 7, a and b, the first two iterations selected variables Chosen Value and Chosen Juice. All variables selected in subsequent iterations provided a marginal explanatory power of <5%. This result was confirmed by the best-subset procedure (Fig. 7c,d). As a single variable, Chosen Value explained more responses than any other variable. As a pair of variables, Chosen Value and Chosen Juice explained more responses than any other pair of variables. Collectively, Chosen Value and Chosen Juice explained 304 responses, corresponding to 90% of responses explained by the whole 24 variables and to 85% of responses that passed the ANOVA criterion.
We thus proceeded with a post hoc analysis, in which we compared the marginal explanatory power of each of these two variables pairwise against that of other, highly correlated variables (Table 5). In this analysis, the explanatory power of Chosen Value was significantly higher than that of both Value Difference and Chosen #. However, the explanatory power of Chosen Value was not statistically higher than that of Total Value, and the explanatory power of Chosen Juice was not statistically higher than that of either Chosen Value A or Chosen Value B. This analysis would thus leave us with some degree of ambiguity.
To gain further insight, we repeated the variable selection analysis on this data set based on spatially invariant responses. In this case, each data point represented a trial type (not a trial typeLR) and linear fits were performed using linear regressions, from which we obtained a slope and the R2. Notably, this procedure is in principle preferable for ACCv, where responses are not spatially selective, because the statistical power is entirely “spent” to identify encoded variables. We obtained results similar to those described above. Across time windows, the one-way ANOVA (factor: trial type) identified 473 task-related responses. Overall, the 19 nonspatial variables included in the analysis explained 433 of 473 (92%) responses. The stepwise method identified Chosen Value, Chosen Juice, and Total Value in the first three iterations, and no other variable reached the 5% criterion in subsequent iterations. The best-subset method revealed that the highest explanatory power was provided by Chosen Value (as a single variable), by Chosen Value and Chosen Juice (as a pair of variables), and by Chosen Value, Chosen Juice, and Max # (as a set of three variables). The partial discrepancy between the results obtained with the two methods was clarified by the post hoc analysis, which revealed the following points. First, the explanatory power of both Chosen Value and Chosen Juice was significantly higher than that of any other challenging variable (all p < 0.005). In particular, the explanatory power of Chosen Value was significantly higher than that of Total Value (p < 10−5). Second, the explanatory power of Max # was statistically indistinguishable from that of Total Value and Total #. In summary, these analyses indicated that the population of neurons in ACCv certainly encoded variables Chosen Value and Chosen Juice. Together, these two variables explained 367 responses, corresponding to 85% of the responses explained by the whole 19 variables, and to 78% of the responses that passed the ANOVA criterion. In addition, a smaller population of neurons in ACCv might encode the Max #, or the Total # or the Total Value. However, we could not disambiguate between these possibilities based on our current data.
Based on these results, we concluded that the population of ACCv indeed encoded variables Chosen Value and Chosen Juice. We thus classified task-related responses from ACCv as encoding one of these two variables. In general, the two variables provided a good account of the population [across variables, mean(R2) > 0.5; Fig. 7e]. In all subsequent analyses of ACCv, we pooled trials for different movement directions.
Subjective value versus physical properties: analysis of U-shaped responses
One major conclusion of the analyses presented above is that we found in both ACCd and ACCv neurons that encode the Chosen Value. Apart from the dissociation between Chosen Value and other value variables (Total Value, etc.), perhaps the most interesting aspect of this result is that neurons in both ACCd and ACCv appear to encode economic value—a subjective quantity that integrates physically distinct dimensions (juice type and juice amount). In this respect, however, one caveat is in order. Although our analysis included many physical variables, it forcedly did not include all possible physical variables. For example, we did not examine the variable Sugar Amount. Thus, to test whether U-shaped responses indeed encode the subjective value as opposed to any physical property of the juices, we conducted the following analysis.
The hypothesis that U-shaped responses encode the Chosen Value leads to a simple prediction. Separating trials in which the animal chose juice A and juice B, we can regress separately the neuronal firing rate against the amounts of juice A and juice B chosen by the animal. The two regressions provide slopes α and β (Fig. 8a,b). If the neuronal response indeed encodes the Chosen Value, then slope α should be proportional to the subjective value of juice A, slope β should be proportional to the subjective value of juice B, and the ratio α/β should be equal to the value ratio. In other words, the slope ratio α/β provides an independent and neuronal measure of the relative value of the two juices, which should be equal to the relative value measured behaviorally from the choice pattern. For the response illustrated in Figure 8a,b, this prediction is met: the relative value measured neuronally (α/β = 2.2 ± 0.6, SD computed with error propagation) is statistically indistinguishable from that measured behaviorally at the indifference point (A/B = 2.6).
The evidence that U-shaped responses indeed encode subjective value as opposed to any physical property of the juices emerges from the following observation. For any given juices A and B, the relative value measured behaviorally may vary to some extent from day to day. For example, on some days, the animal may be very thirsty and may not forego large quantities of juice B to obtain 1A, resulting in a lower relative value. On other days, the animal may be less thirsty, resulting in a higher relative value. If U-shaped responses encode a physical property of the juices, they should be unaffected by this day to day variability. Conversely, if U-shaped responses encode the subjective value, the slope ratio α/β should also vary from day to day matching the variability in relative value measured behaviorally. Our data were consistent with the latter prediction. In Figure 8c, we plotted the neuronal measure of relative value (slope ratio, y-axis) against the behavioral measure of relative value (indifference point, x-axis). Each circle represents one response, and the plot includes all and only U-shaped responses recorded in ACCv with 2/3 fruit punch and 1/3 cranberry juice. A linear regression y = a0 + a1x provided the following estimates (±95% confidence interval): a0 = 0.09 (±0.45) and a1 = 0.99 (±0.66). In other words, the two measures of relative value were statistically indistinguishable.
To test the identity between neuronal and behavioral measures of relative values across the population, we performed an ANCOVA (full model) in which we regressed the slope ratio against the behavioral indifference point and we grouped responses by the juice pair. This analysis was done separately for the two brain areas. For ACCd (six juice pairs; Fig. 8d), we found that the slope ratio depended significantly on the indifference point (p < 10−10) but did not depend on either the juice pair or the interaction (indifference point by juice pair) (both p > 0.07). Specifically, for the relationship between slope ratio and indifference point (main term), we obtained slope = 1.18 (±0.24) and intercept = −0.19 (±0.22) (estimate ± SEM). In other words, the neuronal measure and the behavioral measure of relative value were statistically identical with one another.
For ACCv (seven juice pairs; Fig. 8e), we obtained very similar results. The full model ANCOVA indicated that the slope ratio depended significantly on the indifference point (p < 10−10) but did not depend on either the juice pair or the interaction (indifference point by juice pair) (both p > 0.5). For the relationship between slope ratio and indifference point (main term), we obtained slope = 0.83 (±0.16) and intercept = 0.20 (±0.12) (estimate ± SEM). (Note that the intercept did not differ significantly from zero, p > 0.09.) In other words, the two measures of relative value were statistically identical to one another.
In conclusion, this analysis demonstrates that U-shaped responses indeed encode a subjective quantity–namely, the Chosen Value–as opposed to any physical property of the juices.
Range adaptation in ACCd and ACCv
In a previous study, we found that value-encoding neurons in the OFC adapt to the range of values available in any given session (or trial block). In other words, the same range of neuronal firing rates describes different ranges of values in different behavioral conditions (Padoa-Schioppa, 2009). This adaptation phenomenon ensures in principle a computationally efficient encoding of subjective value. We thus examined whether the representations of subjective value in ACCd and ACCv also undergo range adaptation.
We first examined the population of 521 Chosen Value responses recorded in ACCd. Consistent with previous reports (Seo and Lee, 2007; Kennerley et al., 2009), their activity could either increase or decrease as a function of the encoded value. Specifically, responses with positive and negative slopes were 310 of 521 (60%) and 211 of 521 (40%), respectively (Fig. 9a). Apart from the sign, the two distributions obtained for positive and negative slopes had equal median (p = 0.84, Wilcoxon's rank-sum test). For all the analyses of range adaptation, we thus rectified negative-slope responses and pooled them with positive-slope responses (Fig. 9b).
The variable selection analyses presented above indicated that the encoding of the variable Chosen Value was roughly linear. In other words, indicating with φ the firing rate, with V the Chosen Value, and with c0 and c1 the intercept and slope of the encoding, respectively, the following held true: φ = c0 + c1V. Having defined the value range ΔV as the range of values chosen by the animal within a session and the minimum value V0 as the lowest value chosen in that session, this relationship can be written as follows: φ = φ0 + Δφ * (V − V0)/ΔV, where φ0 = c0 + c1V0 is the baseline activity and Δφ = c1 * ΔV is the activity range. In this formalism, the hypothesis that neuronal firing rates adapt to the range of values available in any given session corresponds to the hypothesis that neuronal parameters φ0 and Δφ are independent of the value range ΔV.
In our experiments, each neuron was recorded in one session with one value range (ΔV). However, the range of chosen values varied from session to session, roughly twofold between 4 and 8 uB. We thus divided neuronal responses in two groups depending on whether the range values chosen in that session was ΔV ≤ 6 uB or ΔV > 6 uB. If neuronal firing rates indeed adapt to the value range, then the slope of the encoding c1 should be inversely proportional to the value range ΔV. Consistent with this prediction, we found that the average slope was significantly higher when neurons were recorded with ΔV ≤ 6 uB compared with when neurons were recorded with ΔV > 6 uB (p < 10−10, ANOVA; Fig. 9c).
The most direct evidence for neuronal adaptation came from a population analysis of the firing rate as a function of the Chosen Value. For this analysis, we normalized each response by subtracting the baseline activity φ0 corresponding to the minimum chosen value V0. The entire population of 521 responses is illustrated in Figure 9d, where the two colors label responses recorded with ΔV ≤ 6 uB and ΔV > 6 uB, respectively. In general, activity ranges (y-axis) varied substantially across the population. However, when we averaged the firing rates separately for the two groups of responses neuronal adaptation appeared evident (Fig. 9e).
We also examined the distributions of activity ranges (Δφ) measured for the two groups of responses. Consistent with range adaptation, the two distributions were statistically indistinguishable (p = 0.10, Kruskal–Wallis test; Fig. 9f). One possible concern is that this last result may be due to saturation or ceiling effects, as opposed to adaptation. However, it can be noted that the two activity traces in Figure 9e are well separated throughout the spectrum of chosen values, and both traces are close to linear. This observation indicates that the fact that activity ranges are independent of the value range (Fig. 9f) is genuinely due to range adaptation.
We repeated these same analyses for the population of 256 Chosen Value responses recorded in ACCv, and we obtained very similar results (Fig. 10). In ACCv, responses with positive and negative slopes were 172 of 256 (67%) and 84 of 256 (33%), respectively. The two distributions had equal median (p = 0.69, Wilcoxon's rank-sum test; Fig. 10a). Thus, we rectified negative-slope responses and pooled them with positive-slope responses (Fig. 10b). We separated neuronal responses in two groups depending on whether they had been recorded in sessions with value range ΔV ≤ 6 uB or ΔV > 6 uB, and we examined the slope of the encoding as a function of the value range. Consistent with range adaptation, the average slope was significantly higher when neurons were recorded with ΔV ≤ 6 uB compared with when neurons were recorded with ΔV > 6 uB (p < 10−10, ANOVA; Fig. 10c). We then examined the neuronal firing rate directly as a function of the Chosen Value. We normalized responses by subtracting the baseline activity, and we plotted the entire population of 256 responses (Fig. 10d). In general, activity ranges (y-axis) varied substantially across the population. However, when we averaged the firing rates separately for the two groups of responses, neuronal adaptation appeared evident (Fig. 10e). The distributions of activity ranges measured for the two groups of responses were statistically indistinguishable (p = 0.65, Kruskal–Wallis test; Fig. 10f). Importantly, the activity traces for the two groups of responses (Fig. 10e) were well separated throughout the spectrum of chosen values, indicating that the result in Figure 10f was indeed due to neuronal adaptation as opposed to saturation or ceiling effects.
In conclusion, our analyses indicated that value-encoding neurons in both ACCd and ACCv undergo range adaptation. In both areas, neuronal responses encoding the Chosen Value adapted in such a way that a given range of firing rates represented different ranges of values in different behavioral conditions.
Partial adaptation on the timescale of individual trials
The results illustrated in Figures 9 and 10 demonstrated range adaptation on the timescale of behavioral sessions, which in this study typically included 200–400 trials. One interesting question concerns the time course of the adaptation process. In our experiments, monkeys learned the range of values included in each session by performing in the task. Thus, adaptation presumably took place over the course of multiple trials at the beginning of each session. In the following analysis, we examined whether and to what extent adaptation could be observed on a trial-by-trial basis.
We used the same approach previously undertaken for the analysis of OFC. In our experiments, different trial types were randomly interleaved so that the Chosen Value varied from trial to trial. The hypothesis that neurons undergo range adaptation leads to a simple prediction. Consider for example the cell in Figure 11a, recorded in a session in which the Chosen Value varied between 2 and 6 uB. Consider now only trials in which the monkey chose 3B (Chosen Value = 3 uB). These trials could follow trials in which the Chosen Value was <3 uB; alternatively, they could follow trials in which the Chosen Value was >3 uB. If the neuron indeed undergoes adaptation, then its activity should depend at least slightly on the value chosen in the previous trial. Specifically, its activity should be slightly elevated when the value chosen in the previous trial was <3 uB; it should be slightly depressed when the value chosen in the previous trial was >3 uB.
To test this prediction more generally, we divided trials depending on whether the Chosen Value in the current trial (trial n) was higher or lower than the Chosen Value in the previous trial (trial n − 1). Indicating with V(k) the value chosen in trial k, the two groups of trials were thus defined as V(n) > V(n − 1) and V(n) < V(n − 1). Consistent with the prediction, the activity for trials V(n) > V(n − 1) was slightly higher than that for trials V(n) < V(n − 1) (Fig. 11a).
To quantify this effect, we computed for each trial type the “normalized difference,” defined as the difference between the firing rate obtained for trials V(n) > V(n − 1) and the firing rate obtained for trials V(n) < V(n − 1), normalized by the firing rate obtained pooling all trials. We then defined δ as the average normalized difference (average across trial types). Thus, δ represented the average percentage modulation of trial n − 1 on trial n. For the response in Figure 11a, we obtained δ = 13%. Two panels in Figure 11b illustrate the distributions of δ obtained, respectively, for the population of 521 Chosen Value responses recorded in ACCd and for the population of 256 Chosen Value responses recorded in ACCv. Although both areas present a substantial variability, responses with δ > 0 were a significant majority in both areas (both p < 10−4, binomial test). Furthermore, in both areas, mean(δ) was significantly greater than zero (both p < 10−6, t test). Quantitatively, we measured mean(δ) = 5% and mean(δ) = 7% in ACCd and ACCv, respectively. Interestingly, these measures are similar to the one previously obtained for OFC [mean(δ) = 6% (Padoa-Schioppa, 2009)].
To test the reliability of this result, we repeated the analysis for n − 2, n − 3, etc., dividing trials depending on whether the Chosen Value on trial n was higher or lower than the Chosen Value on trial n − 2, n − 3, etc. As expected, the adaptation effect faded rapidly with the trial distance in both areas (Fig. 11c). For a control, we repeated the analysis for n + 1 dividing trials depending on whether the Chosen Value on trial n was higher or lower than the Chosen Value on trial n + 1. We did not expect any effect in this case because neurons should not adapt to future events. Indeed, mean(δ) was statistically indistinguishable from zero.
In conclusion, a small but significant neuronal adaptation could be observed at the timescale of individual trials in both ACCd and ACCv, with an incidence and a time course very similar to those previously measured for OFC (Padoa-Schioppa, 2009).
We reported three main results. First, we found clear physiological differences between the activity of neurons in the dorsal and ventral bank of the anterior cingulate sulcus. Specifically, spatially selective neurons were frequent in ACCd (20%) but nearly absent in ACCv. With respect to the time profile, ACCd was most active during the delay before movement initiation, whereas ACCv was most active at the end of the trial, after juice delivery. Second, we found representations of subjective value in both areas. Specifically, neurons in both ACCd and ACCv encoded variables Chosen Value and Chosen Juice. In addition, neurons in ACCd encoded the direction of the eye movement. In each area, these variables explained a vast majority of neuronal responses and did so significantly better than all other variables examined. Many previous studies concluded that neurons in ACC encode reward (Niki and Watanabe, 1979; Shidara and Richmond, 2002; Ito et al., 2003; Matsumoto et al., 2003; Williams et al., 2004; Amiez et al., 2006; Seo and Lee, 2007). However, this is the first unequivocal demonstration that neurons in these areas represent subjective value as distinguished from the physical properties of the rewards. Third, we found that value-encoding neurons in both areas adapt to the range of values available in any behavioral condition. Range adaptation measured in ACCd and ACCv is qualitatively and quantitatively similar to that previously found in OFC (Padoa-Schioppa, 2009; Kobayashi et al., 2010) and analogous to phenomena observed in other brain systems (Laughlin, 1989; Griffin et al., 2005; Tobler et al., 2005; Clifford et al., 2007; Dean et al., 2008; Bermudez and Schultz, 2010).
A role of ACCd and ACCv in decision making?
From the point of view of economic choice behavior, it is particularly interesting to compare ACCd and ACCv, on the one hand, and OFC, on the other hand. In this respect, we observed three key differences. First and perhaps most important, neurons in OFC encoded both pre-decision variables (Offer Value) and postdecision variables (Chosen Value, Chosen Juice). In contrast, neurons in ACCd and ACCv encoded only post-decision variables. Second, the engagement of OFC in the trial preceded that of ACCd and ACCv. Specifically, responses peaked in the Postoffer time window for OFC, in the Late Delay time window for ACCd, and in the Postjuice time window for ACCv. [Similar findings were previously reported by Kennerley and Wallis (2009b).] Third, the direction of the saccade executed by the animal modulated the activity of neurons in ACCd. In contrast, the activity of neurons in OFC and ACCv did not depend on the visuomotor contingencies of the task. Together, our results indicate that information about the choice outcome (Chosen Value and Chosen Juice) reached ACCd and ACCv after the comparison between values (i.e., the decision) took place in other brain regions. In this respect, it is also interesting to note that, although the activity of neurons in ACCd was spatially selective, we did not find evidence for the encoding of “action value” variables Offer Value L and Offer Value R. In other words, at least from the limited vantage point provided by ACCd, it appears that decisions in our choice task did not involve the comparison of action values. Conversely, the activation timing and the coexistence in ACCd of choice outcome signals and movement direction signals raises the possibility that this area may contribute to the processes through which choice outcomes and subjective values inform the motor systems.
Prima facie, the conclusion that ACCd/v are computationally downstream of economic decisions may appear at odds with previous studies (Walton et al., 2002; Williams et al., 2004; Kennerley et al., 2006; Rudebeck et al., 2008; Camille et al., 2011). Upon closer examination, however, the present results complement well recently emerging notions on the functional role of ACC. In broad strokes, decision tasks can be distinguished depending (1) on whether the task includes or does not include a learning component and (2) on whether options are defined by stimuli and/or by actions. For example, Rudebeck et al. (2008) tested monkeys in two tasks that included a learning component and such that options were defined either only by stimuli or only by actions. A recently emerging notion is that ACC lesions seem to impair performance particularly in tasks that include a learning component, and especially if options are defined exclusively by actions. In contrast, performance is essentially spared in tasks that do not include learning and/or when options are defined by stimuli (Kennerley et al., 2006; Rudebeck et al., 2008). Notably, our economic choice task did not include a learning component. Furthermore, in any given trial, each option was associated with both a stimulus and an action. Thus, the conclusion that ACCd/v does not contribute to decision making in our task is consistent with previous results. In fact, our present observations seem to resonate with recent reports indicating a specific role of ACC in associative learning (Alexander and Brown, 2011; Kennerley et al., 2011; Wallis and Rich, 2011). It can be noted that variables Chosen Value and Chosen Juice encoded in ACCd/v, if measured in a learning context, could provide the quantities necessary to evaluate performance and thus compute a teaching signal. In this view, the role of ACC in decision making would be indirect: by influencing learning, the activity of this areas in any given trial would affect decisions in subsequent trials.
Possible roles of ACCd and ACCv in other cognitive processes
Our results are also relevant to two hypotheses previously put forth on the functional role of ACC. First, based mostly on imaging data, several authors proposed that neural activity in ACC generally reflects the cognitive conflict or the difficulty of the task (Botvinick et al., 2001; Paus, 2001; Ridderinkhof et al., 2004; Rushworth et al., 2004). In our experimental paradigm, the decision difficulty can be operationally identified with the variable Value Ratio (i.e., Oth/Ch Value). This variable has its maximum at the indifference point (Value Ratio = 1), where the two values are very similar and thus the decision is presumably difficult. As one “moves away” from the indifference point, the Value Ratio becomes smaller and the decision becomes presumably easier. In the extreme cases of forced choices (Value Ratio = 0), the decision is trivial. Consistent with this operational definition, we previously observed in a version of the task that did not impose a delay between offer and movement initiation that reaction times were highly correlated with the Value Ratio (Padoa-Schioppa et al., 2006). With this premise, current results appear in contrast with the cognitive conflict hypothesis because vanishingly few neurons encoded the Value Ratio in either area. Notably, our recordings sampled densely and extensively both banks of the cingulate sulcus. Thus, the negative result vis-à-vis decision difficulty appears credible. Interestingly, other studies searched and failed to find individual cells encoding cognitive conflict or task difficulty in the primate ACC (Ito et al., 2003; Nakamura et al., 2005). The discrepancy between single-cell and imaging studies may possibly be due to anatomical mismatches between species (e.g., human ACC include cell types not found in macaques) or perhaps to the imperfect correspondence between hemodynamic responses and neuronal spiking activity (Botvinick, 2007). In any case, the cognitive conflict hypothesis did not find support in our data.
Second, in a recent study, Hayden et al. (2009) reported that neurons in the ACC encode fictive (counterfactual) reward signals. In their experiment, monkeys selected one among eight possible targets, seven of which delivered a small amount of juice and one of which (the optimal target) delivered an amount of juice randomly variable but on average much higher than that of the other targets. On any given trial, the position of the optimal target was most likely (60% of trials) the same as in the previous trial. Thus, the optimal strategy was to select on each trial the target located in the same position occupied by the optimal target in the previous trial. If on trial n the monkey selected the optimal target, the optimal strategy on trial n + 1 was to select the same target; if on trial n the monkey selected a different target, the optimal strategy on trial n + 1 was to switch and select the position occupied by the optimal target on trial n. Since the amount of juice delivered by the optimal target varied randomly from trial to trial, switching in the latter case was optimal independently of the amount “missed” on trial n (the fictive reward). However, curiously, the fictive reward significantly affected switching in both animals. Remarkably, Hayden et al. (2009) found in ACC neurons encoding the fictive reward size as well as the actual reward size. In our experiments, fictive rewards are measured by the variable Other Value, which accounted for very few responses in our data sets. So how can our results be reconciled with those of Hayden et al. (2009)? In this respect, it can be noted that, in the study by Hayden et al. (2009), fictive rewards were behaviorally relevant because they guided target selection in the next trial. In other words, the task included a learning component that was based on both the location and the amount of the fictive reward. Thus, one possibility is that neurons recorded by Hayden et al. (2009) did not encode fictive reward per se (i.e., Other Value), but rather encoded the value of stimuli that guided the learning process. This hypothesis predicts that fictive reward signals in ACC should disappear if they do not guide learning. In this view, our conclusion that neurons in ACCd and ACCv encode the subjective value of chosen goods might extend to the subjective value of other behaviorally relevant goods. Similar considerations apply to OFC, where very few responses encoded the Other Value in our experiments (Padoa-Schioppa and Assad, 2006, 2008), whereas a recent study found fictive reward signals in a learning context (Abe and Lee, 2011) [see discussion in the study by Padoa-Schioppa and Cai (2011)].
In conclusion, we found two independent representations of subjective value in ACCd and ACCv. The general roles of these representations in various cognitive functions remain somewhat unclear. Our results suggest that these two areas do not contribute to economic decisions per se. At the same time, our results complement well recent work indicating a specific role of ACC in associative learning. More generally, the coexistence in ACCd of choice outcome signals and movement direction signals suggests that this area may represent a gateway through which the choice system informs motor systems.
This work was supported by National Institute of Mental Health Grant R00-MH080852 and The Whitehall Foundation Grant 2010-12-13. We thank Jordan McCall for substantial help in animal training and in preliminary data collection. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Camillo Padoa-Schioppa, Department of Anatomy and Neurobiology, Washington University in St. Louis, 660 South Euclid Avenue, Campus Box 8108, St. Louis, MO 63110.
- Abe and Lee, 2011.↵
- Alexander and Brown, 2011.↵
- Amiez et al., 2006.↵
- Bermudez and Schultz, 2010.↵
- Botvinick, 2007.↵
- Botvinick et al., 2001.↵
- Camille et al., 2011.↵
- Clifford et al., 2007.↵
- Dean et al., 2008.↵
- Dunn and Clark, 1987.↵
- Glantz and Slinker, 2001.↵
- Griffin et al., 2005.↵
- Hayden and Platt, 2010.↵
- Hayden et al., 2009.↵
- Hillman and Bilkey, 2010.↵
- Ito et al., 2003.↵
- Kable and Glimcher, 2009.↵
- Kennerley and Wallis, 2009a.↵
- Kennerley and Wallis, 2009b.↵
- Kennerley et al., 2006.↵
- Kennerley et al., 2009.↵
- Kennerley et al., 2011.↵
- Kobayashi et al., 2010.↵
- Koyama et al., 2001.↵
- Laughlin, 1989.↵
- Lee et al., 2007.↵
- Matsumoto et al., 2003.↵
- Matsumoto et al., 2007.↵
- Nakamura et al., 2005.↵
- Niki and Watanabe, 1979.↵
- Padoa-Schioppa, 2009.↵
- Padoa-Schioppa, 2011.↵
- Padoa-Schioppa and Assad, 2006.↵
- Padoa-Schioppa and Assad, 2008.↵
- Padoa-Schioppa and Cai, 2011.↵
- Padoa-Schioppa et al., 2006.↵
- Paus, 2001.↵
- Rangel and Hare, 2010.↵
- Ridderinkhof et al., 2004.↵
- Rudebeck et al., 2008.↵
- Rushworth and Behrens, 2008.↵
- Rushworth et al., 2004.↵
- Rushworth et al., 2007.↵
- Sallet et al., 2007.↵
- Seo and Lee, 2007.↵
- Shidara and Richmond, 2002.↵
- Tobler et al., 2005.↵
- Wallis and Kennerley, 2010.↵
- Wallis and Rich, 2011.↵
- Walton et al., 2002.↵
- Williams et al., 2004.↵