Abstract
Perceptual decisions are made when sensory evidence accumulated over time reaches a decision threshold. Because decisions are also guided by prior information, one important factor that is likely to shape how a decision is adaptively tuned to its context is the predictability of forthcoming events. However, little is known about the mechanisms underlying this contextual regulation of the perceptual decision-making process. Mathematical models of decision making predict two possible mechanisms supporting this regulation: an adjustment of the distance to the decision threshold, which leads to a change in the amount of accumulated evidence required to make a decision, or a gain control of the sensory evidence, leading to a change in the slope of the sensory evidence accumulation. Here, we show that predictability of the forthcoming event reduces the distance to the threshold of the decision. Then, combining model-driven fMRI and the framework of information theory, we show that the anterior cingulate cortex (ACC) adjusts the distance to the decision threshold in proportion to the current amount of predictive information and that the dorsolateral cortex (DLPFC) codes the accumulation of sensory evidence. Moreover, the information flow from the ACC to the DLPFC region that accumulates sensory evidence increases when optimal adjustment of the distance to the threshold requires more complex computations, reflecting the increased weight of ACC's regulation signals in the decision process. Our results characterize the respective contributions of the ACC and the DLPFC to contextually optimized decision making.
Introduction
Recent advances in neuroscience and mathematical psychology have begun to unravel the neurobiological mechanisms underlying decision making (Gold and Shadlen, 2007). Perceptual decision making, the ability to select a specific action based on our perception, proceeds from the integration of sensory evidence to a categorical choice between alternatives (Smith and Ratcliff, 2004; Lo and Wang, 2006; Bogacz, 2007a). In sequential sampling models, this gradual gathering of sensory information favoring a particular choice is defined as a drift of an abstract decision variable toward a decision threshold. A choice is made when a decision variable is equal to its decision threshold (Carpenter and Williams, 1995; Hanes and Schall, 1996; Usher and McClelland, 2001). These mathematical models of decision making received renewed interest after the demonstration by monkey electrophysiological studies that perceptual choices are made when the ramping activity of neural populations in the dorsolateral prefrontal cortex (DLPFC) and the lateral intraparietal (LIP) area reaches a given threshold (Hanes and Schall, 1996; Kim and Shadlen, 1999; Huk and Shadlen, 2005; Hanks et al., 2006). The ramping rate of this neural activity, which represents the accumulation of sensory evidence, correlates with the decision variable predicted by sequential sampling models. In humans, fMRI studies confirmed the involvement of a similar DLPFC–intraparietal network in coding the decision variable (Heekeren et al., 2004; Forstmann et al., 2008; Ivanoff et al., 2008; Tosoni et al., 2008; van Veen et al., 2008).
One important factor that is likely to shape how a decision is adaptively tuned to its context is the predictability of the forthcoming event (Luce, 1991; Dayan and Abbott, 2001; Harrison et al., 2006; Doya, 2008). However, it remains unclear how decision making is modulated by this predictive information at both the behavioral and the neural levels. Sequential sampling models predict two mechanisms that modulate the decision based on contextual information (Carpenter and Williams, 1995; Reddi et al., 2003): (1) An adjustment of the distance to the decision threshold, which leads to a change in the amount of evidence required to make a decision, but no variation in the slope of the decision variable. According to this mechanism, higher predictability of forthcoming events would reduce the distance to the decision threshold (Fig. 1a, top panels). (2) An adjustment of the gain of sensory evidence, leading to a change in the slope of the decision variable, but not in the distance to the threshold. According to this hypothesis, higher predictability would increase the slope of the decision variable (Fig. 1a, bottom panels).
Hypothetical mechanisms by which predictability of forthcoming events modulates the decision-making process. a, Left panels illustrate the progressive drifts of decision variables toward their respective decision threshold. They also correspond to the activity predicted by the decision model for neural populations accumulating sensory evidence. A decision is made when a decision variable equates its threshold. Right panels illustrate how the reciprobit analysis of RT distributions reveals distinct regulatory mechanisms of the decision process. Threshold modulation hypothesis: Reciprobit lines swivel toward lower RT when predictability increases (top right panel), reflecting the lowering of the decision threshold (top left panel). Gain control hypothesis: Reciprobit lines shift toward lower RT when predictability increases (bottom right panel), reflecting the faster rise of the decision variable toward the decision threshold (bottom left panel). b, Task design. Participants had to identify a target shape out of three shapes by pressing a response button (red stars, ISI = 1.35 ± 0.76 s SD). Unbeknownst to the subjects, the next shape could be predicted on the basis of recent history. In first-order sequences, only the last trial had a predictive value on the next shape, whereas in second-order sequences, both the last and the penultimate trials had a predictive value on the next shape. c, Example of a set of transition rules from a first-order sequence. Arrows represent transitions from one shape to the next one and transition probabilities are indicated nearby each arrow.
Here, we manipulated the amount of contextual information available to predict which stimulus is going to appear next (Fig. 1b). This allowed us to distinguish between these two hypotheses by characterizing the computational mechanisms underlying the effect of predictability on decisions. Then, having found that predictability modulates the distance to the threshold of the decision and not the gain control of sensory evidence, we identified the brain regions involved in this regulation, as well as those coding the decision variable. Finally, we investigated how changes in effective connectivity between these distributed brain regions lead to contextually optimized perceptual decisions.
Materials and Methods
Subjects.
Fourteen healthy right-handed subjects [8 males, mean age (±SD): 25.14 ± 3.37 years, mean right-handedness score as estimated by the Edinburgh scale (±SD): 0.86 ± 0.1, mean level of higher education (±SD): 3.6 ± 2.2 years] participated in the study (Oldfield, 1971). None of the participants showed any past or current neurological or psychiatric conditions, as assessed by a medical interview and all had normal or corrected-to-normal visual acuity. None of them was on medication at the time of the study. The experiment was approved by the local ethics committee. Subjects gave their written informed consent and underwent standard medical exams before participation.
Perceptual decision-making paradigm.
Participants performed a GO/NO-GO task, in which they had to press a response button for a specific target shape (presented at the beginning of each sequence) among three possible shapes (Fig. 1b). Each participant performed the perceptual decision task on 12 randomly ordered unique sequences. All sequences consisted of the successive presentation of blue shapes (circle, square, or triangle) displayed at the center of a screen. At the beginning of each new sequence, the participant was shown one of the three shapes on a yellow background. This shape was the target for the current sequence. After 5 s of target display, the background turned black and the perceptual decision task began. Participants were instructed to press a response button held in their right hand each time they identified the current target, as quickly and as accurately as possible. Each sequence was composed of 400 successive stimuli presented for 300 ms every 400 ms (Fig. 1b). A fixation cross was presented for 10 s between two successive sequences. Unbeknownst to participants, there were two types of sequences (Fig. 1b): in first-order sequences, the next shape was conditioned on the last shape, whereas in second-order sequences, the next shape was conditioned on the last two shapes. Figure 1c shows a set of transition rules for a first-order sequence. Using the framework of Shannon's information theory, we computed for each decision the surprise (Eq. 1), which measures how unlikely an event is, and the predictive information on the forthcoming stimulus (Eq. 2–3), which measures how much the knowledge of the recent history (last shape or penultimate shape) reduces this surprise. Statistical transition rules were held constant within a sequence and varied between sequences. Moreover, both first- and second-order sequences were selected to fall into three categories based on their mutual information (first-order sequences: Eq. 4; second-order sequences: Eq. 5): zero (low), one-third (medium), and two-thirds (high) of the maximum theoretical mutual information (with a tolerance margin of 5%, Eq. 6). This procedure guaranteed a broad range of predictive information values during the experiment.
To minimize potentially confounding effects classically observed during sequential choices, statistical transition rules were constrained to ensure a low repetition probability (prepetition < 0.05) and to minimize tandem repeats in sequences (Kornblum, 1969). Moreover, the pace of perceptual decisions was chosen in accordance with the psychophysical literature, which shows that the behavioral effect of surprise on response time (RT) is minimized when repetition probability is low and the interval between two perceptual decisions is short (Kornblum, 1969) and was further adjusted to guarantee a high level of accuracy (>90%). Sequences were selected to ensure average frequencies in the 0.05–0.45 range for each stimulus, thus controlling for oddball effect (Ranganath and Rainer, 2003) by ensuring that sequences did not contain rare events. All stimuli occurred with the same probability over the whole experiment. Finally, for each sequence, we selected the most sparsely distributed shape in the range 0.25–0.4 as the target. At the end of the scanning session, participants were systematically asked about “their awareness of regularities” as in Harrison et al. (2006). Only one subject reported he had noticed a pattern, once, during the experiment but could not give a specific example.
Working hypothesis.
Bayesian formulations of perceptual decision making distinguish between the prior information (before observing the stimulus) and the accumulation of evidence in term of likelihood (during stimulus observation). In these formulations, the quantity accumulating evidence starts at different levels, according to the prior information. Evidence is then accumulated at a constant rate until the criterion is reached. In the context of our design, before the stimulus arrives, the predictive information (prior beliefs) will reset the level of activity and therefore change the distance to the decision threshold. From this perspective, the predictive information is a prior and the information conveyed by the stimulus represents the evidence entailed by its likelihood. Thus, our hypothesis was that both behavioral and fMRI data would be better explained by an adjustment of the distance to the decision threshold in proportion to the predictive information on the forthcoming stimulus than by a modulation of the slope of the decision variable.
Note that in our paradigm, we kept the level of sensory information constant (by using exactly the same three stimuli across the experiment). This does not mean that what is being integrated in the current paradigm is not sensory evidence. Indeed, perceptual decisions occur even when visual categorization may appear “unambiguous” while monkeys make saccades toward a target. For example, frontal eye field and lateral prefrontal cortex neurons exhibit a ramping activity that decrease after reaching a threshold value (Kim and Shadlen, 1999), both when manipulating the position (Hanes and Schall, 1996) or the color (Stanford et al., 2010) of unambiguous targets.
Thus, although information about local stimulus-response predictability is manipulated in the current study, it is not “integrated” over the decision process (what is being integrated is still sensory evidence). This approach mixing local stimulus-response predictability and perceptual decision making distinguishes our study from the neuroimaging literature investigating which brain regions encode measures of information theory, such as surprise and uncertainty (Huettel et al., 2005; Strange et al., 2005) or their influences on EEG components or corticospinal excitability (Bestmann et al., 2008; Mars et al., 2008).
Estimates of surprise and predictive information.
In Shannon's information theory, the surprise of an event is defined by the current estimate of its marginal log-probability (abbreviated as ut in Eq. 1). This measure has been considered as an instantaneous measure of the level of saliency (Harrison et al., 2006). For each new shape et, displayed at time step t, the current estimate of the surprise (ut) is defined in the following way:
The predictive information of the upcoming event is an instantaneous measure of the loss of uncertainty about its occurrence due to the knowledge of the previous event(s) (also called “surprise reduction”). This last measure quantifies the amount of information available at a given time to predict the outcome of the ongoing perceptual decision and is poorly correlated with the surprise (a high level of predictive information does not necessarily mean that surprise is low). We computed both the predictive information conveyed by the last event (abbreviated as p1,t in Eq. 2) and by the last two events (abbreviated as p2,t in Eq. 3). For each new shape et, displayed at the time step t, current estimates of the predictive information (p1,t and p2,t) are defined in the following way:
Supplemental Figure S1 (available at www.jneurosci.org as supplemental material) illustrates the trial-to-trial fluctuations of the predictive information conveyed by the last (Eq. 2) and by the last two (Eq. 3) shapes over the course of two exemplary sequences.
The average predictive information over a whole sequence of events is called the mutual information. By analogy with the predictive information, we computed the mutual information conveyed by the last event (abbreviated as Im1,t in Eq. 4) and by the last two events for each sequence (abbreviated as Im2,t in Eq. 5). Mutual information is maximum when a sequence is entirely determined (abbreviated as Immax in Eq. 6). It is noteworthy that predictive information is an event-bound measure, whereas mutual information pertains to the average predictability in a sequence without relating to any specific event.
where k is the number of different shapes in a sequence.
Because participants learned the statistical structure of the sequence as stimuli were presented, we used a simple Bayesian learning scheme (an ideal Bayesian observer), in which all marginal and conditional probability estimates were updated after each new event. Our ideal Bayesian observer was initialized with flat prior distributions and was reset at the beginning of each new sequence to account for the lack of prior knowledge on the upcoming sequence (Harrison et al., 2006). For each new shape et, presented at time step t, current values of the marginal probability of the event i (Eq. 7) and of the joint probability of two successive events i and j (Eq. 8) and of three consecutive events i, j, and k (Eq. 9) are defined in the following way:
where ni,j,kt is the number of triplets i, j, k at time step t; and ni,jt is the number of duplets i, j at time step t.
We computed the surprise (Eq. 1) and the predictive information (Eqs. 2, 3) at each time step using the estimates provided by Equations 7–9.
Multilinear model of response times.
Behavioral analyses were performed using the software packages R and Statistica (v7.1). We defined the error rate as the number of missed targets divided by the total number of targets over each sequence. Response times were calculated as the time elapsed between the onset of a target and the subject's response.
First, we searched for the best multilinear model of the observed RT using a descending strategy. The error rate, the surprise, the predictive information conveyed by the last shape and by the last two shapes (abbreviated respectively as p1 and p2), as well as all the first-order interactions between these explanatory variables were included in the “full” model. Akaike information criterion was minimized after the surprise and all first-order interactions were removed from the “full” model (βsurprise = −0.015, p = 0.126).
In the reduced behavioral model (Eq. 10), RTs are modeled as a weighted sum of explanatory variables in which the standardized parameter estimates of the model, such as βp1 and βp2, are referred to as “behavioral” sensitivity because they represent the slope between response times and the amount of predictive information conveyed by the last and the penultimate shape. So, estimated βs correspond to the independent contribution of each explanatory variable to the prediction of RT. This multilinear model is consistent with the relationship predicted by sequential sampling models of decision in which error rate and response times depend on the amount of predictive information available, while keeping the level of sensory information constant across trials. Note that, by construction of our design, there was no predictive information conveyed beyond the penultimate event available to predict the forthcoming event.
LATER model: how RT distributions are used to distinguish between the two modulation mechanisms of the decision process.
In the LATER (linear approach to threshold with ergodic rate) model (Carpenter and Williams, 1995; Reddi and Carpenter, 2000; Reddi et al., 2003), the onset of a stimulus (e.g., a shape) is followed by the linear rise of a signal (decision variable) from a starting point (red dashed line) to a decision threshold D (Fig. 2a, blue line). A response is initiated when the decision signal reaches the threshold. On different trials, the slope of the decision signal varies randomly, but is distributed as a Gaussian probability density function with mean slope s and with standard deviation sd (Hanes and Schall, 1996). So, according to the LATER model, the distribution of RT reflects the projection of the decision variable on the decision threshold and depends on three parameters: the distance to the threshold (difference between the starting point and the decision threshold), the mean slope, and its standard deviation (as stated in Eq. 11 and Fig. 2).
LATER model. a, When a stimulus is presented, a decision signal rises linearly from the starting point (dashed red line) with an average slope s and a trial-to-trial standard deviation sd. When the signal reaches the decision threshold, a motor response is initiated. In the model, the amount of sensory evidence needed to reach a decision is represented by the difference between the starting point and the decision threshold, representing the distance to the threshold D (Reddi and Carpenter, 2000). b, Relationship between RT distributions, reciprobit plots, and LATER model parameters. The reciprobit plot represents the cumulative 1/RT distribution, linearized by computing z-scores (probit transformation), as a function of RT. This graphical representation of RT distribution has two important features: (1) the intercept with the x = ∞ line solely depends on the mean slope of the decision process and is equal to s/sd; (2) the intercept with the y = 0 line is s/D (which depends on both the slope and the distance to the threshold).
Equation 11 simply expresses that, under the LATER model, the main decision process yield 1/RT following a normal distribution, with mean s/D and with standard deviation sd2/D2. In addition, fast guesses are modeled as an additional normal distribution, whose mean is equal to zero, and its own standard deviation.
Since 1/RT of the main process is normally distributed (Eq. 11), it is possible to compute z-scores that express the divergence of the observed 1/RT from the median 1/RT. Plotting z-scores of 1/RT's cumulative distribution against RT plotted on a reciprocal time axis yields a straight line, which is called a reciprobit plot (as illustrated in Fig. 2b). This graphical representation is useful because the resulting line intersects z-score = 0 at the median latency s/D, which depends on both the mean slope (s) and the distance to the threshold (D), whereas it intersects RT = ∞ at a point that does not vary with the distance to the decision threshold (Fig. 2b). Importantly, the mathematical properties of the reciprobit transformation provide us with a graphical representation that distinguishes between the two modulation mechanisms in the LATER model (see Fig. 1a): (1) if the modulation mechanism is an increase of the slope (sensory evidence gain control), then both intersects will vary in the same proportion and the line will shift (Fig. 1a, lower right panel); and (2) if the modulation mechanism is a decrease of the decision threshold, then only the z-score = 0 intersect will vary, which will result in a swivel of the reciprobit line around the RT = ∞ intersect (Fig. 1a, upper right panel).
To summarize, the reciprobit transformation directly allows us to derive the z-scores of 1/RTs cumulative distribution from the RT distribution. From these z-scores, it is then possible to estimate the parameters of the LATER model (distance to the decision threshold, mean slope, and sd of the slope) that best fit the data and to perform a Bayesian statistical test to identify the mechanisms of regulation that best explain the changes between conditions (decision threshold modulation or gain control of the sensory evidence).
Psychophysics: LATER model and reciprobit plots.
To assess the mechanism underlying the effect of predictive information on decision, we performed a standard reciprobit analysis (Carpenter and Williams, 1995; Reddi et al., 2003).
First, we normalized each participant RT dataset to the population's average and standard deviation. Then, we pooled all the RT datasets together and collapsed the behavioral data from first- and second-order sequences using the optimal amount of predictive information. Next, we discretized each participant RT dataset into equal bins and excluded from further analysis those that did not contain enough data to allow for reliable fits of the decision model. This constraint led us to exclude the 5% lowest predictive information values from further analyses. This is because, in our experiment, the distribution of predictive information had a long tail toward low values. By the end of these preprocessing steps, we had sorted RT data into 6 bins with continuously increasing levels of predictive information ([−0.43, −0.05, 0.25, 0.62, 1, 1.32] bits).
Then, we performed a reciprobit transformation on the resulting RT distributions. This transformation is based on the LATER model and makes testable predictions about how RT distributions should change according to two different modulation mechanisms: distance to the decision threshold or sensory evidence accumulation rate (Carpenter and Williams, 1995; Gold and Shadlen, 2007) (Fig. 1a). Plotting the reciprobit lines, which are linearized cumulative RT distributions plotted on a reciprocal time scale, highlights those changes.
In addition to this qualitative assessment of the mechanism regulating the decision process, we used a Bayesian model selection strategy to identify the regulation mechanism that most likely explained the changes observed in RT distributions across levels of predictive information. To do so, we fitted a LATER model using a standard simplex minimization routine and a likelihood-based cost function under the hypotheses that changes in RT distribution either resulted from changes in the sensory evidence accumulation rate or resulted from changes in the distance to the threshold. Model comparison was performed by fitting the LATER model for each experimental condition in such a way that either the slope or the distance to the decision threshold was fixed across condition, depending on the hypothesis tested. Finally, we computed the log likelihood ratio between the two hypotheses (LDT − LGain, difference between the log likelihood of the distance to the threshold modulation, LDT, and the log likelihood of the gain control mechanism, LGain) and used the cutoff value of the Bayesian factor (LDT − LGain ≥ 2.3) (Jeffrey, 1998) to assess the significance level of our result. For example, a log likelihood ratio equal to 4.6 indicates that a modulation of the distance to the threshold is 100 times more likely than a gain control of the sensory evidence (e4.6 = 100).
To assess whether our finding that predictive information modulates the decision threshold depended on some specific aspect of the LATER model, we also fitted a Ratcliff's drift-diffusion model (RDM) to our data using the “D-mat” toolbox (Vandekerckhove and Tuerlinckx, 2008) (http://ppw.kuleuven.be/okp/software/dmat/). However, because the RDM has been specifically formulated for two-alternative choices, it was not possible to use its standard formulation. Because our experiment is a GO/NO-GO task, subjects only responded to one target, which implies that there were no RT distributions for false-negative trials (no response for a GO trial) and for true-negative trials (no response for a NO-GO trial). Therefore, we adapted the D-mat toolbox to fit a GO/NO-GO version of the RDM on our data: first, we modified the loss function to only fit a “hit” RT distribution (Vandekerckhove and Tuerlinckx, 2008); second, we fixed the relative position between the starting point and the boundaries, which means that the changes in the distance from the starting point to the boundary were a priori attributed to the boundary parameter. Overall, our version of the RDM (“single boundary” RDM) retained from the version implemented in the D-mat toolbox the drift-diffusion mechanism, the upper decision boundary, the explicit account of nondecision time (and of its variability), and the variability in the starting point. This version of the RDM was adequate because we only estimated parameters relating to “hit” RT distributions. Note that if we had investigated errors RT distributions and error rates, an implicit lower boundary would also have been necessary (Gomez et al., 2007; Ratcliff and McKoon, 2008).
With these modifications of the D-mat toolbox, it was possible to reliably retrieve the distance to the decision threshold and the slope parameters from synthetic RT datasets. Moreover, we assessed the ability of the “single-boundary” RDM to correctly identify the modulation mechanism underlying changes between conditions using only the hit RT distribution and a Bayesian selection strategy [Bayesian information criterion (BIC); smaller values mean a better model in terms of goodness of fit and parsimony]. In the slope condition (a synthetic RT dataset simulating a change in the slope of the accumulation of evidence), the model in which the drift rate parameter was set free between conditions had the best Bayesian information criterion (BIC drift rate = 41,162, BIC boundary = 55,167). In the threshold condition (a synthetic RT dataset simulating a change in the distance to the threshold), the model in which the boundary parameter was set free between conditions had the best Bayesian information criterion (BIC drift rate = 43,441, BIC boundary = 43,075).
Finally, we performed a Bayesian selection analysis among drift-diffusion models instantiating three alternative mechanisms (distance to the decision threshold, nondecision time, average slope of diffusion process) on our own RT dataset.
fMRI data acquisition.
Subjects were scanned at the CERMEP - Imagerie du Vivant using a research dedicated 1.5 T MRI scanner (Siemens Magnetom Sonata with an eight-channel head coil). We acquired 800 echo-planar T2*-weighted functional volumes (200 volumes/run, 4 runs) per experiment. Each volume comprised 28 slices acquired continuously over 2.65 s (TE = 60 ms; interleaved acquisition; slice thickness 4 mm; 0.4 mm noncontiguous; parallel to the subject's Sylvian fissure plane; angle to AC–PC: 20–30°; in-plane resolution: 3.44 × 3.44 mm2; matrix size: 64 × 64), allowing complete brain coverage. Additionally, T1-weighted images were acquired at the end of each experiment (MP-RAGE: TR = 1970 ms; TE = 3.93 ms; T1 = 1100 ms; resolution: 1 × 1 × 1 mm3; matrix size: 256 × 256). Head motions were minimized using foam padding and headphones with earplugs were used to dampen the scanner noise.
fMRI data preprocessing.
Data preprocessing was performed using the Statistical Parametric Mapping software (SPM2b, Wellcome Department of Imaging Neuroscience, University College London, UK, www.fil.ion.ucl.ac.uk/spm). The first three volumes of each run were removed to allow for T1 equilibrium effects (197 volumes/run). Before statistical analysis, we applied a slice-timing correction using the time center of the volume as reference. Then, head motion correction was applied using rigid-body realignment. We used realignment parameters during the statistical analysis as covariates to model out potential nonlinear head motion artifacts. Functional and morphological images were then normalized into standard MNI space using SPM's default templates. Finally, functional volumes were resampled and smoothed with an 8 mm FWHM Gaussian kernel. A 256 s temporal “high-pass filter” regressor set was included in the design matrix to exclude low-frequency noise and artifacts.
Finally, we explored the data for potential artifacts using tsdiffana, mean and variance images (http://imaging.mrc-cbu.cam.ac.uk/imaging/DataDiagnostics). An artifact is defined as the co-occurrence of a variance spike and a mean intensity drop uncorrelated with experimental design. Only the last two volumes of one participant's session met these criteria and were modeled as confounds in the design matrix. Translational movements estimated during the realignment procedure were small as compared to the voxel size (<1 mm).
General linear model 1: main fMRI data statistical analysis.
Whole-brain statistical parametric analyses were performed using a two-stage random-effect approach. We estimated independently the model parameters from each subject's dataset and then made population inferences using the parameter intersubject variance. Regressors of interest were constructed by convolving functions representing the events with the canonical hemodynamic response function. Three event-related categorical regressors (“stimulus regressor,” “decision-related regressor,” and “motor regressor”) and three parametric regressors (surprise, predictive information conveyed by the last shape, and predictive information conveyed by the last two shapes) were used to model the events occurring during the sequences (Fig. 3).
Regressors included in the statistical analysis of fMRI data. The main GLM included three categorical and three parametric regressors (see Materials and Methods, General linear model 1: main fMRI data statistical analysis). The three categorical regressors modeled the main steps of perceptual decision making: sensory processing, decision-related activity and motor response. Three parametric regressors were derived from the decision-related regressors and hierarchically orthogonalized. These parametric regressors modeled the modulation of BOLD activity at the time of decision by the surprise and the predictive information conveyed by the last and the penultimate shape.
(1) The first regressor modeled the visual stimulation as 0.3-s-long boxcar functions time locked to the onset of visual stimuli (referred to as the “stimulus regressor”).
(2) The ongoing processes during perceptual decision formation (referred to as the “decision-related regressor”) were modeled as boxcar functions convolved with the response time duration, time locked to each target onset. Because this condition pooled the decision-related activity regardless of the context in which it took place, it modeled the part of the decision-related activity not modulated by its context. Three parametric regressors were added to the decision-related regressor to account for the effect of surprise (Eq. 1) and predictive information (Eqs. 2, 3) on the decision process. These parametric regressors were hierarchically orthogonalized in the following order: surprise, predictive information conveyed by the last shape only, and predictive information conveyed by the last two shapes. This orthogonalization hierarchy naturally emerged from the mathematical definitions of the parameters (Büchel et al., 1998) and unambiguously separated the effect of the information conveyed by the last shape from the information conveyed by the penultimate shape into two parametric regressors. To build these regressors, we weighted each event of the decision-related regressor by the current, and continuously updated, estimates of the parameters, so that each event was characterized by its own set of parameter values.
(3) Finally, the last categorical regressor modeled the motor response associated with the button press, and was modeled as a Dirac function using the timing of the button press as onset. Thus, our model explicitly separated the motor-related activity from the decision-related activity.
Statistical inferences were performed with a threshold of p < 0.05 (clusterwise) familywise error (FWE) corrected across the whole brain (p < 0.001 voxelwise) (see supplemental Tables S1, S2, available at www.jneurosci.org as supplemental material).
Correlation between “neural” and “behavioral” sensitivity to predictive information.
We reasoned that blood oxygenation level-dependent (BOLD) activity in a brain region modulating the distance to the threshold should be predictive of each participant's RT variations (Figs. 4, 5). Thus, we performed a correlation analysis between the sensitivity to predictive information estimated from brain activity and the sensitivity to predictive information estimated from response times for both the information conveyed by the last and the penultimate shape.
Higher predictive information reduces the distance to the decision threshold. a, RT decreased as predictive information increased in first-order [left panel: rp1 = −0.295 (orange), rp2 = −0.273 (red), both p < 10−6] and second-order sequences [middle panel: rp1 = −0.235 (orange), rp2 = −0.292 (red), both p < 10−6]. During second-order sequences (middle panel), RTs were better correlated with the predictive information conveyed by the last two shapes (red) than with the predictive information conveyed by the last shape only (orange) but not during first-order sequences (left panel), indicating that all available predictive information was used in the regulation of the decision process. Finally, there was no effect of surprise (right panel, green) on RT (rsurprise = −0.02, p = 0.126). b, Reciprobit plot based on pooled RT from all participants showing a swivel toward lower RT when predictive information increases, as hypothesized in Figure 1a (upper right panel). This aspect is confirmed by the log likelihood ratio (LDT − LGain), in accordance with the hypothesis of the modulation of the distance to the threshold. c, Distance to the decision threshold as a function of the level of predictive information available. Error bars represent 95% confidence intervals of the distance to the threshold. The color code represents the same levels of predictive information in both panels (from −0.43 to 1.32 bits).
Event-related response in the ACC predicts individual ability to use predictive information to modulate the distance to the threshold. a, Parametric response to the amount of predictive information conveyed by the last shape (rendered with a threshold of p < 10−3 uncorrected, activations surviving a threshold of 5% clusterwise corrected across the whole brain are circled in red). The color scale represents the slope of the decrease in activity for an increasing amount of predictive information conveyed by the last shape. Note that it does not reflect deactivation. Also note that additional brain regions (not shown here) also survived the statistical threshold used and are listed in supplemental Table S1 (available at www.jneurosci.org as supplemental material). b, Scatter plots of correspondence between “neural” and “behavioral” sensitivities to predictive information in the ACC (n = 14). For each participant, the two sensitivity measures link event-related responses in the ACC and modulation of RTs. (See Materials and Methods, Correlation between “neural” and “behavioral” sensitivity to predictive information.) Individual differences in “behavioral” sensitivity to predictive information conveyed by the last shape (left) and the penultimate shape (right) were predicted by individual differences in “neural” sensitivity in the ACC. Higher “behavioral” sensitivity to predictive information directly reflects the ability to modulate the distance to the threshold. c, Scatter plots of correspondence between BOLD signal change in the ACC and the distance to the decision threshold (left panel) or the gain of the sensory evidence (right panel). Each point represents the BOLD signal change in the ACC plotted against the distance to the decision threshold estimated using the LATER model averaged over the four levels of predictive information (−0.3, 0.18, 0.72, 1.22 bits) for each subject.
To measure the “behavioral” sensitivity to predictive information, we fitted the multilinear model of RT previously identified to each individual RT set, thereby estimating its βs (Eq. 10). Here, the βs are measures of the slope of the decrease in response time with increasing predictive information conveyed by the last and the penultimate shape, regardless of the current accuracy level. This analysis yielded a behavior-based measure of the individual ability to use the predictive information conveyed by the last shape and the penultimate shape to modulate the distance to the threshold. Then, to measure the “neural” sensitivity to predictive information, we extracted for each participant, and in every brain region found to be sensitive to predictive information in the main fMRI analysis [region of interest (ROI)-based approach using MarsBaR toolbox v0.38, p < 0.001 voxelwise; see below, ROI analyses], the β estimates of the parametric regressors, which provided us with measures of the slopes of the decrease between event-related BOLD activity with increasing predictive information conveyed by the last shape and the penultimate shape.
Finally, we performed nonparametric correlation analyses between individual “behavioral” and “neural” sensitivity to identify the brain regions in which the slope of the relationship between predictive information and BOLD activity was predictive of the slope of the relationship between predictive information and RT (Spearman's correlation) (see supplemental Table S1, available at www.jneurosci.org as supplemental material).
General linear model 2: controlling for potential confounding effects in the anterior cingulate cortex.
To assess the specificity of our fMRI findings, we performed an additional statistical parametric analysis, in which we added to the general linear model (GLM) 1 (see Figs. 3, 6) three parametric regressors to the “decision-related” regressor, orthogonalized in the following order: the first four parametric regressors controlled for the effects of error likelihood, prediction error, entropy, and surprise, whereas the following two parametric regressors modeled the modulation of BOLD signal by the predictive information conveyed by the last shape and the predictive information conveyed by the penultimate shape. This procedure ensured that any potential confounding effect from the error likelihood, prediction error, entropy, and surprise were removed from the estimation of the effects of the predictive information parametric regressors.
Whole-brain analysis of parametric responses to entropy, surprise, error likelihood and prediction error. a, Statistical maps are rendered with a very lenient uncorrected threshold of p = 0.01 to illustrate the absence of effect of these potential confounds in the ACC. Left and right sagittal views are shown in the left and right columns. The cold color scale represents negative correlations and the hot color scale represents positive correlations. b, ROI-average parametric response in the ACC to surprise (U), error likelihood (Error), prediction error (TD), and entropy (H). None of the four parametric regressors explained a significant portion of the BOLD activity in the ACC (NS, not significant).
The error likelihood parametric regressor was computed for each sequence from participant error rates during target trials. The error prediction parametric regressor (δt) was computed using a standard Rescorla–Wagner algorithm (Dayan and Abbott, 2001), whose learning parameter (α) was adjusted to maximize the correlation between participants RTs and Probt(et = i|et−1 = j), the reinforcement learning estimate of the conditional probability of a shape (et) at the time step t given the last shape (et−1) (Eqs. 12, 13). Finally, Bt is a binary function equal to 1 when the expected event actually occurs (et = i) and to 0 if it does not (et ≠ i) (Eq. 13). The best fit of the Rescorla–Wagner algorithm was obtained for a learning rate α = 0.08 (range explored 0.01–0.15).
Then, the entropy parametric regressor was computed for each shape from Equation 14. The entropy is classically viewed as an information-theoretic equivalent to the concept of conflict (Berlyne, 1957).
General linear model 3: correlation between BOLD activity and LATER model parameters.
To assess the correlation between LATER model parameters and BOLD activity, we built and estimated a second variant of GLM 1, in which we sorted the events previously included in the “decision-related” regressor (Fig. 3) into four discrete levels of predictive information ([−0.3, 0.18, 0.72, 1.22] bits), which divided the range of predictive information into bins of equal size (see above, Psychophysics: LATER model and reciprobit plots). Each bin included enough data to reliably perform individual fits of the LATER model.
Then, using these four levels of predictive information, we built four distinct categorical regressors, in which each event was modeled using a Dirac function time locked on the onset of the target. These four categorical regressors replaced the “decision-related” regressor of GLM 1 (Fig. 3). GLM 1 and GLM 3 were otherwise identical.
This procedure allowed us to perform nonparametric correlation analyses (Spearman's correlation) between BOLD activities at the time of decision averaged over the four levels of predictive information for each participant and the corresponding averaged LATER model's parameter estimates (Figs. 5c, 7c) (see below, ROI analyses).
ROI analyses.
We extracted ROI average of estimated βs for the three parametric regressors included in GLM 1 and for the four categorical regressors modeling the levels of predictive information in GLM 3. To do so, ROIs were built from functional clusters from GLM 1 (p < 0.001, voxelwise) intersected with a 6-mm-radius sphere centered on the cluster's peak voxel using the MarsBaR toolbox (v0.38, http://marsbar.sourceforge.net).
Conjunction analysis.
We performed a conjunction analysis testing the conjunction null (Nichols et al., 2005), using SPM2b to identify clusters that exhibited significant negative parametric effects for predictive information conveyed by the last and the penultimate shape at the onset of decisions. However, because conjunction tests are not as sensitive as single-contrast testing for the average effect over all contrasts and thus underestimate the underlying effect (Friston et al., 2005), and because we had a strong a priori hypothesis regarding the involvement of the DLPFC in implementing the decision variable, here inferences were performed with a level of significance of p < 10−3 uncorrected (Fig. 7).
Brain regions coding the decision variable. a, Conjunction map showing the brain regions activated during perceptual decision making in which BOLD activity is negatively modulated by the amount of predictive information conveyed by the last and the penultimate shape. We rendered our map using an uncorrected threshold of p < 0.001 (level of significance used for inference, red voxels) and a threshold of p < 0.005 to show the full extent of the activations (yellow voxels). b, Average parametric response to surprise (u) and predictive information (p1 and p2) in these brain regions. The parametric response to the predictive information conveyed by the last shape (p1) and the penultimate shape (p2) was not significantly different (NS) in any of the regions identified (p1 = p2, orange and red bars; rIPS: p = 0.43; rDLPFC: p = 0.34). There was no parametric response to surprise (u = 0, green bars; rIPS: p = 0.29; rDLPFC: p = 0.8). c, Scatter plots of correspondence between BOLD signal change in the ACC and accumulation's slope average, for each of the brain regions shown in Figure 7a (circled in red; see Materials and Methods, General linear model 3: correlation between BOLD activity and LATER model parameters). Each point represents the BOLD signal change in the ACC and the slope of sensory evidence accumulation estimated using the LATER model averaged over the four levels of predictive information (−0.3, 0.18, 0.72, 1.22 bits) for each subject (see supplemental Fig. S3, available at www.jneurosci.org as supplemental material).
Structural equation modeling.
First, to characterize functional subdivisions between the anterior and posterior DLPFC, we built a morphological ROI of the DLPFC (WFU PickAtlas atlas v2.4, http://fmri.wfubmc.edu, dilatation parameter = 2 voxels, bilateral mask including BA9, BA10, and BA46 from WFU PickAtlas built-in atlas, volume = 5994 voxels) and computed statistical maps of the parametric effect of the predictive information conveyed by the last shape and the penultimate shape (p < 0.001 voxelwise). From this analysis, we isolated four functional subregions whose activity reflected the amount of predictive information at the moment of the decision (p < 0.05 clusterwise, SVC): left anterior DLPFC, right anterior DLPFC, left posterior DLPFC, and right posterior DLPFC.
Then, we extracted ROI-averaged time series during first- and second-order sequences for each participant from 6-mm-radius spheres centered at the peak voxel of the four brain regions identified in the DLPFC (Fig. 8, dashed white circles) and the anterior cingulate cortex (ACC) (Fig. 8, plain white circle) (N = 5124 volumes for each brain region and condition, no missing values or deleted data). Structural equation modeling was performed using the Mx software package (v1.65b). Figure 8 represents the path diagram as arrows to indicate directional or symmetric connections between the functional regions included in the model. We performed a maximum-likelihood-based estimation of the model path coefficients on the correlation matrix derived from the two resulting time series and statistical inferences on path coefficient variations between the first- and second-order sequences using a nested model approach (supplemental Table S3, available at www.jneurosci.org as supplemental material) (no convergence problems or inadmissible solutions).
Diagram of effective connectivity between ACC and DLPFC. DLPFC subregions in which BOLD signal decreased as the predictive information conveyed by the last shape increased are rendered in blue, DLPFC subregions in which BOLD signal decreased as the predictive information conveyed by the penultimate shape increased are rendered in green and DLPFC subregions in which both effects were present are rendered in red (p < 0.005 uncorrected, for display). Red cluster corresponds to the DLPFC subregion coding the decision variable shown in Figure 7. The plain white circle represents the ACC, which is buried within the medial wall of the frontal cortex. The structural equation model included oriented path (arrows) connecting the ACC and the four functional subregions found in the DLPFC. Dashed circles white indicate the location and the extent of the spheres used for time series extraction. A yellow arrow indicates a significant increase of the path coefficient between first-order and second-order sequences, whereas a black arrow indicates a significant decrease of the path coefficient (all p < 10−2). Finally, white arrows indicate path coefficient variations that are not significant. Variations of effective connectivity from first-order sequences to second-order sequences are indicated as relative variations next to each path (supplemental Table S3, available at www.jneurosci.org as supplemental material, indicates absolute values and statistical significance).
The overall model fit was assessed with standard goodness of fit indices, all indicated a good quality of fit (normed fit indices = 0.91, centrality index = 0.9, and relative noncentrality indices = 0.91; index values above 0.9 indicate a good quality of fit) (Mueller, 1996).
Functional connectivity analysis.
To identify brain regions that were functionally coupled with the ACC, we assessed the correlation between BOLD activity in this “seed” region and BOLD activity in each voxel of the brain. To do so, we extracted the cluster-averaged time course from the functional cluster we found in the ACC (Fig. 5a) (ROI-based approach using MarsBaR toolbox v0.38, p < 0.001 voxelwise; see above, ROI analyses) and included this time course as a regressor not convolved with a hemodynamic response function in a GLM. This GLM also included a 256 s low-pass filter and head motion parameters as regressors of noninterest. We then computed group-level SPM using the standard SPM's RFX approach. Supplemental Figure S6 (available at www.jneurosci.org as supplemental material) shows the main result of this analysis with a threshold of 5% voxelwise, FWE corrected across the whole brain.
Results
Psychophysics: predictive information reduces the distance to the threshold of the decision
RT decreased linearly as predictive information increased (Fig. 4a, left and middle panels), showing that participants successfully used the statistical structure of sequences to predict the forthcoming shape. Moreover, participants adjusted to the actual structure of the sequences (first or second order) to exploit all the predictive information available. Indeed, RTs were better correlated with the predictive information conveyed by the last two shapes (last shape and penultimate shape, rp2) (Fig. 4a, red line) than with the predictive information conveyed by the last shape only (rp1) (Fig. 4a, orange line) during second-order sequences, but not during first-order sequences (Fig. 4a) (Hotelling's t, first-order sequences, rp1 = rp2; p = 0.98; second-order sequences, rp1 = rp2; p < 10−6). We also assessed the contribution of the source (last or penultimate shape) of predictive information on decision response time by fitting a multilinear model to all participants' RTs (see Materials and Methods, Multilinear model of response times). Predictive information had the same influence on RT whether it was conveyed by the last shape (βp1 = −0.148 ± 10−2, p < 10−5) or by the penultimate shape (βp2 = −0.148 ± 10−2, p < 10−5; βp1 = βp2, t = −3.97 ± 10−4, p = 0.49), showing that the efficiency of the modulation did not depend on the source of the information. This decrease in RT with increasing predictive information did not occur at the cost of accuracy, as shown by a factorial analysis crossing the type of sequence (first order or second order) and the predictive information averaged over each sequence (supplemental Fig. S2, available at www.jneurosci.org as supplemental material; Eqs. 4, 5). Finally, there was no effect of surprise on RT in our experiment, as expected from previous literature (Fig. 4a, right panel) (Kornblum, 1969).
Next, to identify which of the two predicted mechanisms—modulation of the distance to the threshold or gain control of the sensory evidence—mediated the effect of predictive information on decision making, we fitted a LATER model to the RT distribution of the subjects' responses and compared the likelihood of the two modulation mechanisms (Carpenter and Williams, 1995; Reddi et al., 2003). The modulation of the distance to the threshold by predictive information was significantly more likely than a gain control of the sensory evidence [log likelihood ratio, defined as the difference between the log likelihood of the distance to the threshold modulation hypothesis (LDT) and the log likelihood of the gain control hypothesis (LGain), LDT − LGain = 148.42, which is “decisive” according to Bayesian inference theory] (Jeffrey, 1998). Moreover, individual model fits showed that all the participants used predictive information to modulate their distance to the decision threshold, except for two participants for whom data did not allow to conclusively select a mechanism over the other (supplemental Fig. S3, available at www.jneurosci.org as supplemental material). Then, we performed a reciprobit analysis of the population's RT distribution (linearization of RT cumulative distribution resulting in “reciprobit lines”) (see Materials and Methods, Psychophysics: LATER model and reciprobit plots, and Fig. 4b). This analysis provided us with a graphical representation of the mechanism modulating decision RT based on the variations of the reciprobit line for increasing amounts of predictive information: if the distance to the threshold decreases, then the line swivels around an intercept point toward lower RT (as in Fig. 1a, top right panel). By contrast, if the slope of the decision variable increases, then the line shifts toward lower RT (as in Fig. 1a, bottom right panel). The swivel of the reciprobit line with increasing levels of predictive information observed in Figure 4b further confirmed the reduction of the distance to the threshold by higher predictive information (supplemental Fig. S3, available at www.jneurosci.org as supplemental material). Finally, we observed a strong negative correlation between the distance to the threshold and predictive information (r = −0.995, p < 10−6) (Fig. 4c).
These results did not depend on specific features of the LATER model since fitting a drift-diffusion model to our dataset also led to the conclusion that predictive information modulates the distance to the threshold (log likelihood ratio, LDT − LGain = 89.051). Furthermore, there was an excellent agreement between the distance to the threshold estimated using the LATER and the drift-diffusion models for all levels of predictive information (r = 0.99, p = 10−6).
Thus, our behavioral results demonstrate that the effect of predictive information on decision RT is mediated by the modulation of the distance to the decision threshold, not by gain control, and uses all the predictive information available to minimize decision RT.
Brain network responding to predictive information
In parallel with our behavioral results showing faster RTs with increasing predictive information (Fig. 4a), we investigated the relationship between decision-related brain activity and predictive information (see Materials and Methods, General linear model 1: main fMRI data statistical analysis; and Fig. 3). The results revealed a negative correlation between predictive information conveyed by the last shape and the BOLD activity in the ACC, the inferior frontal gyri bilaterally, the right intraparietal sulcus region (IPS), and the DLPFC bilaterally (p < 0.05 clusterwise corrected for multiple comparisons across the whole brain) (see Fig. 5a and supplemental Table S1, available at www.jneurosci.org as supplemental material).
These patterns of decision-related activity were preserved when adding prediction errors, error likelihood, entropy (which is a proxy for conflict), and surprise as potential confounds in a new analysis, supporting the specificity of the relationship between BOLD activity in all these brain regions and predictive information (supplemental Table S2, available at www.jneurosci.org as supplemental material; and see Materials and Methods, General linear model 2: controlling for potential confounding effects in the anterior cingulate cortex). This additional analysis excludes alternative interpretations of the ACC's response in terms of conflict monitoring, postdecisional prediction errors, and error monitoring (Fig. 6; supplemental Fig. S4, available at www.jneurosci.org as supplemental material) (Holroyd and Coles, 2002; Botvinick et al., 2004; Brown and Braver, 2005).
The anterior cingulate cortex modulates the distance to the threshold of the decision
Within the brain regions showing a parametric response to predictive information (supplemental Table S1, available at www.jneurosci.org as supplemental material), we then assessed whether individual differences in brain activity during decision making predicted individual differences in the ability to exploit predictive information to reduce response time (see Materials and Methods, Functional connectivity analysis). From our behavioral analyses showing that the modulation of the distance to the threshold results in a linear decrease of RT with increasing predictive information (Fig. 4), we predicted that in the brain regions modulating the distance to the threshold, individual differences in “neural sensitivity,” defined as the slope of the decrease in event-related activity as predictive information increased, should predict “behavioral sensitivity,” i.e., the slope of the decrease in RT as predictive information increased. The ACC was the only brain region in which individual differences in event-related response (“neural” sensitivity) predicted each individual's ability to use the information available to modulate the distance to the threshold (“behavioral” sensitivity) (Fig. 5b). Moreover, this link between ACC's function and modulation of the distance to the decision threshold was further supported by the positive correlation between ACC's BOLD activity and distance to the decision threshold (r = 0.625, p = 0.017) (Fig. 5c, left panel), but not between ACC's BOLD activity and the slope of the accumulation of sensory evidence (r = 0.081, p = 0.785) (Fig. 5c, right panel). Together, these results demonstrate that the ACC is involved in adjusting the distance to the threshold in proportion to the current amount of predictive information.
The dorsolateral prefrontal cortex codes the decision variable
In a next step, we took advantage of basic properties of sequential sampling models to identify the brain regions computing the decision variable. First, assuming a coupling between neuronal firing rates and BOLD activity, we predicted that the BOLD response in the brain regions coding the decision variable should increase with slower decision RT and decrease when predictive information increases (i.e., when the distance to the threshold decreases). This hypothesis is based on the observation that the duration of the ramping neuronal activity coding the decision variable predicts RT and that its height correlates with the distance to the threshold (as illustrated in Fig. 1a, top left panel) (Hanes and Schall, 1996; Huk and Shadlen, 2005). Second, paralleling our behavioral results on RTs, the influence of predictive information on the BOLD response should not depend on the information source (last or penultimate shape) and there should be no influence of surprise on the BOLD response. Finally, BOLD response in brain regions coding the decision variable should reflect the slope of sensory evidence accumulation.
A conjunction analysis between brain regions showing decision-related activity decreasing with higher predictive information conveyed by both the last and the penultimate shapes isolated the anterior part of the right DLPFC and the right IPS (p < 0.001 uncorrected) (Fig. 7a). As expected, BOLD activity in these brain regions was identically modulated by the predictive information conveyed by the last and by the penultimate shape (Fig. 7b) (paired t test, p1 = p2, orange and red bars; right IPS: p = 0.43; right DLPFC: p = 0.34), and there was no influence of surprise on neural activity in these brain regions (Fig. 7b) (t test, u = 0, green bars; rIPS: p = 0.29; rDLPFC: p = 0.8).
Among these two brain regions, we assessed the correlation between BOLD response and the slope of sensory evidence accumulation (see Materials and Methods, General linear model 3: correlation between BOLD activity and LATER model parameters). Indeed, although the strength of sensory evidence was kept constant throughout the experiment, there were fluctuations of the slope of sensory evidence accumulation between subjects, as can be seen in supplemental Figure S3 (available at www.jneurosci.org as supplemental material). These individual fluctuations of the slope of sensory evidence accumulation correlated with BOLD activity in the right DLPFC (r = 0.64, p = 0.016), but not in the right IPS (r = 0.09, p = 0.75), thereby strongly supporting the involvement of the DLPFC in coding the decision variable (Fig. 7c).
Effective connectivity between the anterior cingulate cortex and the dorsolateral prefrontal cortex
Finally, having characterized the complementary computations performed in the ACC (Figs. 4, 5), which modulates the distance to the threshold, and the DLPFC region coding the decision variable (Fig. 7), we investigated whether the effective connectivity from the ACC to this DLPFC region increased when optimal regulation of the distance to the threshold required more complex computations. We formalized our hypothesis as a structural equation model (path diagram represented with arrows connecting the ACC to the DLPFC in Fig. 8), based on known anatomical pathways between the ACC and the DLPFC (Beckmann et al., 2009) and an ROI analysis of the parametric effect of predictive information in the DLPFC (Fig. 8). When comparing first-order to second-order sequences, a situation in which computation of the optimal threshold adjustment increases in complexity, the path coefficient from the ACC to the region of the DLPFC that codes the decision variable (Fig. 8, right anterior DLPFC's activation, x, y, z: 45, 45, 12) increased significantly, which was not the case for path coefficients along the other paths originating from the ACC (Fig. 8; supplemental Table S3, available at www.jneurosci.org as supplemental material). Interestingly, this effect was paralleled by an increase in the information flow from right anterior to posterior DLPFC region (right posterior DLPFC activations, x, y, z: 39, 6, 27).
Discussion
The accuracy of a perceptual decision depends on the amount of sensory evidence accumulated (Gold and Shadlen, 2007). However, gathering evidence takes time, which results in a tradeoff between a decision's speed and the accuracy achieved. Thus, optimal decision making should exploit all sources of information available, taking advantage of both the sensory evidence extracted from the environment and the knowledge of contingencies built upon past experiences (Bogacz, 2007b; Gold and Shadlen, 2007). Here, we showed that humans effectively use the predictability of forthcoming events to modulate the distance to the threshold of their decisions, substituting predictive information for sensory information in the decision process to speed up action selection without loss of accuracy (Fig. 4; supplemental Fig. S2, available at www.jneurosci.org as supplemental material). Remarkably, people both estimate and use predictive information optimally, adjusting to environmental dynamics of varying complexity.
The key novel finding reported in this study is the coding in the ACC of a signal reflecting the adjustments of the distance to the threshold in proportion to the current amount of predictive information. This pivotal role of the ACC in the contextual guidance of the decision process is supported by two lines of evidence: (1) neural sensitivity to predictive information in the ACC accurately predicts individual fluctuations in the ability to use predictive information to modulate the distance to the threshold of the decision (Fig. 5); and (2) effective connectivity from the ACC to the DLPFC region accumulating sensory evidence increases when optimal adjustment of the distance to the threshold requires more complex computations, reflecting the increased weight of ACC's regulation signals in the decision process (Fig. 8). Overall, our results strongly support the idea that contextually optimized decisions arise from the integration of complementary computations performed in a network of specialized brain regions. In this conceptual framework, the ACC's main function is the computation of regulation signals that optimally adjust the distance to the threshold to the context.
The involvement of the DLPFC in the accumulation of sensory evidence is supported by the fact that BOLD activity observed in this brain region (1) increased with slower decision response times, (2) was negatively modulated by the amount of predictive information conveyed by the last and the penultimate shape, (3) did not depend on the source of predictive information (last or penultimate shape), and (4) is correlated with the slope of the decision variable (Fig. 7). This finding extends previous reports that the DLPFC accumulates sensory evidence related to the correct choice (Kim and Shadlen, 1999; Heekeren et al., 2004; Philiastides and Sajda, 2006; Philiastides and Heekeren, 2009). Note that we implicitly referred to sensory evidence accumulation as the computational mechanism by which a decision variable is implemented, but we acknowledge that other mechanisms have been proposed and are also possible (Ditterich, 2006; Cisek et al., 2009). Finally, the DLPFC activity we observed cannot be attributed to higher attention at the time of target appearance because this would have predicted increased BOLD response with higher predictive information (the latter being associated with faster RTs in our task). By contrast, we observed a negative correlation between predictive information and BOLD signal in this brain region (Fig. 7).
Previous fMRI studies reported a relationship between choice uncertainty and activity in the medial prefrontal cortex when subjects learn through trials and errors the probability of making a correct choice (Volz et al., 2003; Huettel et al., 2005; Volz et al., 2005; Grinband et al., 2006; Huettel, 2006; Platt and Huettel, 2008). These findings parallel studies using fMRI in humans or brain lesions in monkeys showing that one of ACC's critical functions is to build and update an extended action/reward history to guide future decisions optimally (Hampton et al., 2006; Kennerley et al., 2006; Behrens et al., 2007). Our results draw an important link between these two fields of research by showing that the ACC is involved in the regulation of the decision-making process using predictive information (a measure of the reduction of uncertainty estimated on the basis of the history of associations between successive events) and suggests that adjustment signals of the distance to the threshold in the ACC may be a general computational mechanism for the contextual guidance of decisions. Interestingly, theoretical insights into representational learning suggest that a learning signal is needed to support such a function (Williams and Goldman-Rakic, 1998; Holroyd and Coles, 2002; Friston, 2003; Dreher et al., 2006; D'Ardenne et al., 2008). The midbrain activation we observed concomitant with the ACC activation could serve such a functional role since prediction error signal has previously been found in the midbrain (although this cluster did not survive correction for multiple comparison, p < 0.001 uncorrected) (see Fig. 5a) (Dreher et al., 2006; Behrens et al., 2007; D'Ardenne et al., 2008).
Previous accounts of the ACC's function have stressed factors other than the contextual regulation of the decision-making process, such as the monitoring of errors and conflicts (Carter et al., 1998; Botvinick et al., 2004; Ridderinkhof et al., 2004), the likelihood of errors (Brown and Braver, 2005), and the role of postdecisional prediction-error signals (Holroyd and Coles, 2002). However, none of these alternative functions could account for the relationship observed here between ACC activity and predictive information. Indeed, additional fMRI analyses of our data showed that both the likelihood of error and the prediction error failed to explain our BOLD activity in the ACC at the time of decision formation (Fig. 6). Moreover, once controlled for the level of predictive information, BOLD activity in the ACC did not significantly differ between slow and fast responses, which rules out interpretations of our ACC activity in terms of conflict monitoring or spurious correlation with RT, which would have predicted that decisions with longer RTs are associated with greater levels of conflict and with higher level of ACC activity (supplemental Fig. S5, available at www.jneurosci.org as supplemental material). Moreover, in our experiment, the entropy, which has been proposed as a direct measure of conflict (Berlyne, 1957) did not account for a significant part of BOLD activity in the ACC (supplemental Table S2, available at www.jneurosci.org as supplemental material; Fig. 6).
It should be noted that ACC's regulatory function of the distance to the threshold does not necessarily imply that this brain region directly implements the threshold of the decision. In fact, a number of theoretical accounts propose that the basal ganglia implement a gating mechanism that signals, by a phasic increase of activity in the direct pathway, the moment when the activity of cortical neurons coding the decision variable crosses the decision threshold (Lo and Wang, 2006; Bogacz, 2007a, 2009; Frank et al., 2007). This phasic increase of activity would cancel the tonic inhibition exerted by the basal ganglia's output nuclei on motor command centers (Redgrave et al., 1999). Despite a current lack of direct evidence, this proposal emphasizes the potentially central role of cingulostriatal projections in conveying contextual regulation signals from the ACC to the main input structure of the basal ganglia (Kunishio and Haber, 1994; Lo and Wang, 2006). Supporting this hypothesis, we observed a strong correlation between BOLD activity in the ACC and in the striatum, showing that these two brain regions are functionally coupled when making simple decisions (supplemental Fig. S6, available at www.jneurosci.org as supplemental material; Materials and Methods, Functional connectivity analysis).
Moreover, a recent fMRI study comparing perceptual decisions with cues emphasizing speed or accuracy reported a negative correlation between individual variations of a measure for response caution (ratio between the starting point and the decision threshold) and BOLD activity at the time of the cue in both the pre-SMA and the striatum (Forstmann et al., 2008). Thus, the pre-SMA and the striatum may be involved in motor preparation of fast action when explicitly cued for speed and may implement the global slowing down observed when cueing for higher accuracy. Other recent fMRI studies also explicitly emphasized the speed of the perceptual decision at the expense of its accuracy (Ivanoff et al., 2008; van Veen et al., 2008). By contrast, in our study, the modulation of the decision process relied upon predictive information (conveyed by recent history) on the forthcoming stimulus, a quantity that participants implicitly tracked and updated online. Our findings highlight the role of the ACC in keeping track of past events to build an inner model of contingencies and in adjusting the distance to the decision threshold and address a more general contextwise modulation of the decision process, which did not result in a simple global inhibition or facilitation of action preparation, but in a weighting of each possible outcome of the decision based on its likeliness. Consistent with our proposal, a recent fMRI study showed that individual differences in perceptual decision criterion shifts induced by expected losses correlates with BOLD activity in the ACC. Although the authors did not analyze their data within the framework of sensory evidence accumulation models, their findings indicate that asymmetric category costs may affect perceptual decision making in a similar way to changes in category expectations (Fleming et al., 2010).
In conclusion, combining psychophysics, model-driven fMRI and the framework of information theory, we characterized the influence of predictive information on two basic elements underlying the formation of human perceptual decision (distance to the threshold and decision variable). Our results reveal how these elements are coded in the human brain and shed a new light on the respective functions played by the DLPFC and the ACC in perceptual decision making. They also suggest new architectural principles governing the organization of the human frontal lobe and how the interactions between the DLPFC and the ACC are required for optimal decision making.
Footnotes
-
This work was funded by a FP6 International reintegration grant and a Fyssen Foundation grant to J.-C.D. P.D. was supported by a French Ministry of Research scholarship and a fellowship from Le Vinatier Hospital (CSRA 05). We thank the CERMEP - Imagerie du Vivant staff for their helpful assistance and Dr. C. Summerfield and E. Koechlin for comments on an early version of the manuscript.
- Correspondence should be addressed to Philippe Domenech, Cognitive Neuroscience Center, Reward and Decision-Making Group, 67 Bd Pinel, 69675 Bron, France. pdomenech{at}isc.cnrs.fr
