Abstract
Curiosity is a basic biological drive, but little is known about its behavioral and neural mechanisms. We can be curious about several types of information. On the one hand, curiosity is a function of the expected value of information, serving primarily to help us maximize reward. On the other hand, curiosity can be a function of the uncertainty of information, helping us to update what we know. In the current studies, we aimed to disentangle the contribution of information uncertainty and expected value of rewards to curiosity in humans. To this end, we designed a lottery task in which uncertainty and expected value of trial outcomes were manipulated independently and examined how neural activity and behavioral measures of curiosity were modulated by these factors. Curiosity increased linearly with increased outcome uncertainty, both when curiosity was explicitly probed as well as when it was implicitly tested by people's willingness to wait. Increased expected value, however, did not strongly relate to these curiosity measures. Neuroimaging results showed greater BOLD response with increasing outcome uncertainty in parietal cortex at the time of curiosity induction. Outcome updating when curiosity was relieved resulted in an increased signal in the insula, orbitofrontal cortex, and parietal cortex. Furthermore, the insula showed a linear increase corresponding to the size of the information update. These results suggest that curiosity is monotonically related to the uncertainty about one's current world model, the induction and relief of which are associated with activity in parietal and insular cortices, respectively.
SIGNIFICANCE STATEMENT Humans are curious by nature. When you hear your phone beep, you probably feel the urge to check the message right away, even though the message itself likely does not give you a direct reward. In this study, we demonstrated that curiosity can be driven by outcome uncertainty, over and above of reward. The induction of curiosity was accompanied by increased activity in the parietal cortex, whereas the information update at the time of curiosity relief was associated with activity in insular cortex. These findings advance our understanding of the behavioral and neural constituents of curiosity, which lies at the core of human information seeking and serves to optimize the individual's current world model.
Introduction
In daily life, we consume an enormous amount of information. Curiosity, defined as “the impulse towards better cognition” (James, 1899), and more contemporarily as “intrinsically motivated information seeking” (Loewenstein, 1994; Oudeyer and Kaplan, 2007) appears a basic biological drive, but little is known about the underlying behavioral and neural mechanisms (Gott-lieb et al., 2013; Kidd and Hayden, 2015). A distinction can be made between instrumental and non-instrumental curiosity, in which instrumental curiosity is the desire to explore and seek information to reach a goal or to maximize rewards in the immediate or longer term (cf. goal-directed exploration) (Daw et al., 2006). Interestingly, also in non-instrumental settings, in which resolution of uncertainty is not useful or even costly, animals still seek information (Bromberg-Martin and Hikosaka, 2009, 2011; Blanchard et al., 2015). For instance, monkeys choose to have outcomes of risky gambles revealed immediately, instead of remaining in a state of uncertainty when waiting for the outcome (Bromberg-Martin and Hikosaka, 2009, 2011). This is the case even though receiving information is not instrumental (i.e., will not help them to improve performance or to obtain higher rewards). Indeed, monkeys are willing to sacrifice a substantial amount (20%–33%) of primary reward to get advance information (Blanchard et al., 2015). Some researchers (Shannon and Weaver, 1949; Dinsmoor, 1983) have highlighted this phenomenon as an exemplary departure from normatively optimal (reward-guided) behavior (but for an alternative account, see Beierholm and Dayan, 2010).
Here, we aimed to elucidate behavioral and neural constituents of curiosity in humans. We designed a lottery task in which two potential sources of curiosity are manipulated independently: outcome uncertainty (OU) and expected value (EV) of the outcome. Consistent with previous experiments on observing behavior (Bromberg-Martin and Hikosaka, 2009, 2011), observing the outcome was designed to not have any impact on received rewards. As such, this passive observation paradigm allowed us to disentangle uncertainty from value accounts of observing behavior.
On each trial, participants received a lottery with an uncertain outcome and associated monetary reward (Fig. 1). We probed participants' curiosity on each trial either explicitly (Experiments 1, 3) or implicitly (Experiment 2) by investigating participants' willingness to wait for the outcome (Kang et al., 2009; Marvin and Shohamy, 2016). We hypothesized that participants are particularly curious in situations of high uncertainty because information then leads to a large belief update (information prediction error). Using fMRI, we examined which brain areas were modulated by OU and EV upon lottery presentation, and their updates upon outcome presentation. We hypothesized that, during lottery presentation, brain regions associated with uncertainty or risk, such as insular cortex (Critchley et al., 2001; Paulus et al., 2003; Preuschoff et al., 2006, 2008; Jepma et al., 2012) and parietal cortex (Huettel et al., 2005; Foley et al., 2017), would be modulated by OU. During outcome presentation, we expected increased activity in brain regions involved in curiosity relief and information updating, such as the insula (Preuschoff et al., 2008; Jepma et al., 2012), orbitofrontal cortex (Jepma et al., 2012; Blanchard et al., 2015), and ventral striatum (Wittmann et al., 2008; Jepma et al., 2012). In addition, we expected activity in reward-related areas (e.g., ventral striatum) to be modulated as a function of EV (O'Doherty, 2004; Knutson et al., 2005) and reward prediction error (RPE) (Schultz et al., 1997; Bayer and Glimcher, 2005; Daw and Doya, 2006).
A, Schematic figure of Experiments 1 and 3. Participants saw a screen on which a vase with 20 marbles was depicted, either of which could be red or blue, and the points associated with these marbles. Participants were told that one of the marbles would be selected for them and that they would be awarded with the points associated with this marble. Next, participants indicated how curious they were about seeing the outcome of the vase (1–4). There was a 50% chance of seeing the outcome, regardless of the participants' curiosity response. Importantly, a marble was selected in every trial, and participants were awarded with the points associated with this marble, also if they would not see the outcome of a trial. For details on the timing of the experiments, see text (for Experiment 1, see Procedure: behavioral; for Experiment 3, see Procedure: fMRI). B, Schematic figure of Experiment 2. The task was similar to Experiments 1 and 3, except that participants indicated whether they wanted to see the outcome of a trial or not. If they responded “Yes,” they had to wait an additional 3–6 s before the outcome was presented to them and if they responded “No,” the outcome was not presented to them. Still, a marble was selected in every trial and participants were awarded with the points associated with this marble, also if they chose not to see the outcome. For details on the timing of Experiment 2, see Procedure: behavioral.
To preview, we found robust increases in curiosity with increasing OU, whereas EV did not robustly modulate curiosity. Curiosity induction through OU generated activity in parietal cortex, whereas its relief was associated with increased activity in a network comprising the insula, orbitofrontal cortex, and parietal cortex. Furthermore, we found a parametric increase in insular activity with increasing information prediction error. Together, these findings suggest that curiosity can be conceptualized as a desire to improve one's current world model and identify neural constituents that accompany this desire.
Materials and Methods
Participants
24 healthy individuals participated in Experiment 1, in which we explicitly probed curiosity (17 women, age 23.1 ± 3.8 years, mean ± SD). Another 25 healthy individuals participated in Experiment 2, in which we implicitly probed curiosity by investigating participants' willingness to wait. One participant was excluded due to a lack of variation in responding (chose to wait in >98% of all trials). Therefore, the final sample of Experiment 2 consisted of 24 participants (17 women, age 24.6 ± 5.5 years, mean ± SD). Finally, 28 healthy individuals participated in Experiment 3, in which we explicitly probed curiosity while non-invasively recording neural activity using fMRI. All participants were right-handed, screened for MR compatibility, and had normal or corrected-to-normal vision. Two participants were excluded due to a technical problem of the MRI scanner. One participant was excluded because of too many missed trials (>10% of all trials), and one participant was excluded due to self-reported difficulties viewing the screen. The final sample for Experiment 3 consisted of 24 participants (18 women, age 24.1 ± 3.2 years, mean ± SD). All three experiments were approved by the local ethics committee (CMO Arnhem-Nijmegen, The Netherlands) under the general ethics approval (Imaging Human Cognition, CMO 2014/288), and the experiments were conducted in compliance with these guidelines. All participants gave written informed consent according to the declaration of Helsinki before participation.
Procedure-behavioral
Experiment 1.
Each trial started with an image of a vase containing 20 marbles (Fig. 1A). Each of the marbles could be either red or blue. In total, four vase configurations were possible: (1) 100% vases: all marbles had the same color; (2) 95%–5% vases: 19 marbles had one color and 1 marble had the other color; (3) 75%–25% vases: 15 marbles had one color and 5 marbles had the other color; and (4) 50%–50% vases: 10 marbles had one color and 10 marbles had the other color. Both colored marbles were associated with points participants could earn. The points for each of the marbles varied between 10 and 90 (in steps of 10). All combinations of points associated with red and blue marbles were possible. The participants were informed that, on each trial, one marble would be selected from the vase and that they would be awarded with the points associated with this marble. The first screen, on which the vase, the marbles, and the points associated with the marbles were depicted, was presented for 3000 ms. Then a blank screen was presented for 500 ms, followed by a response screen during which participants could indicate how curious they were about seeing the outcome of that trial (“How curious are you about the outcome?”). The curiosity scale ranged from 1 to 4. The response screen was presented until the participant responded, with a response limit of 4000 ms. The response screen was followed by a blank screen (500 ms) and an outcome screen (2000 ms). On each trial, participants had a 50% chance that their curiosity would be satisfied by seeing the outcome (curiosity relief) and a 50% chance that the outcome was withheld. This manipulation was explicitly instructed to subjects, and it uncoupled curiosity responses from the actual receipt of the outcome (thus rendering Pavlovian bias accounts of observing behavior less likely, see Beierholm and Dayan, 2010). In the follow-up fMRI experiment (Experiment 3, see below), it enabled us to investigate the neural consequences of curiosity relief (Curiosity Relief–Yes vs Curiosity Relief–No). The outcome screen depicted the vase, the marbles, and points associated with the marbles again, together with a box in which they saw the colored marble that was selected and how many points they earned. When the outcome was not presented, participants saw a black marble instead of a colored marble and question marks at the location of the number of points. This way, the amount of visual input was approximately comparable between presented and not presented outcomes. After a trial ended, there was a blank screen with a jittered duration between 1000 and 2000 ms (uniformly distributed). Importantly, participants had no way of influencing whether they would observe the outcome of a particular trial, or what the outcome of that trial would be. However, participants knew that a marble would be selected in every trial and that they would be awarded with the points associated with that marble, even if the outcome was not presented. They were told that the total amount of points they earned in total would be converted to a monetary bonus at the end of the experiment.
The trials were pseudo-randomized such that the same vase configuration was never presented >4 trials in a row. Each vase configuration was presented on 153 occasions, except for the 100% vases, which were only presented 18 times. These 100% trials were included as a control to check participants' compliance to the task; we expected people not to be curious about trials of which they already knew the outcome. In total, the participants completed 477 trials, divided in 9 blocks of 53 trials. After each block, the participants were instructed to take a short break if they wanted. The experiment lasted ∼75 min in total.
Experiment 2.
In Experiment 2, we aimed to investigate participants' curiosity more implicitly by means of testing their willingness to wait to see the outcome. We used willingness to wait because it is a well-established measure of the motivational value of an item (Frederick et al., 2002), which has been previously linked to curiosity (Kang et al., 2009; Marvin and Shohamy, 2016). The trial setup for Experiment 2 was similar to Experiment 1 (Fig. 1B). The start of the trial was identical, but instead of giving a curiosity response, participants indicated whether they wanted to see the outcome of that trial (“Do you want to see the outcome?”) by pressing either “Yes” or “No.” If they pressed “No,” a blank screen was presented briefly (500 ms), followed by a screen on which the outcome was not presented (2000 ms). If they pressed “Yes,” the blank screen was presented for an additional jittered duration between 3000 and 6000 ms (uniformly distributed) before they saw a screen on which the outcome was presented (2000 ms). The outcome screens looked identical to the outcome screens in Experiment 1. The next trial started after a jittered inter-trial interval between 1000 and 2000 ms (uniformly distributed). The trials were again pseudo-randomized in a way that the same vase configuration was never presented in >4 trials in a row. The total willingness to wait experiment consisted of 261 trials. Each vase configuration was presented on 81 occasions, except for the 100% trials, which were only presented 18 times. The total duration of the experiment depended on for how many trials the participants indicated that they were willing to wait to see the outcome. Again, participants were told that a marble would be selected on every trial and that they would be awarded with the points associated with that marble, also when they decided not to wait to see the outcome. The total amount of points they earned would be converted to a monetary bonus at the end of the experiment.
Experimental design and statistical analysis: behavioral
Experiment 1.
The behavioral analyses of Experiment 1 were performed using MATLAB (The MathWorks, RRID:SCR_001622) and SPSS (RRID:SCR_002865). We investigated whether there was a relationship between OU and the curiosity ratings, as well as between EV and the curiosity ratings. To do so, a value of OU was calculated for each trial by multiplying the Shannon Entropy by the absolute difference between the red (x1) and blue (x2) marble points as follows:
where P(xi) denotes the probability that a marble (i) would be drawn. Thereby, OU reflected a combination of two types of variance. The first type is the variance in the division of the marbles, indicated by the Shannon Entropy. The Shannon Entropy is a measure of the uncertainty of the variable given the probabilities, and it quantifies the amount of information a trial contains (Shannon, 1948; Bestmann et al., 2008). The second type of variance is the variance in points associated with the marbles. This variance is indicated by the absolute difference between the points associated with both marbles. In turn, EV values were calculated (Eq. 2) by the sum of the probability that a red marble would be drawn (p1) multiplied by the number of points associated with the red marble (x1) and the probability that a blue marble would be drawn (p2) multiplied by the number of points associated with the blue marble (x2) as follows:
Thereby, EV reflects the EV of reward contained in a trial. We investigated whether there was a relationship between OU and/or EV and the curiosity ratings using a Univariate GLM with dependent variable Curiosity Rating (1–4) and random factors Participant, OU, and EV.
To determine whether OU is a better predictor of curiosity than entropy or absolute difference alone, we computed correlations between OU and curiosity, absolute difference and curiosity, and entropy and curiosity. To formally assess whether both absolute difference and entropy contribute to curiosity, we performed a model comparison in which we compared a model in which absolute difference, entropy, and EV were included as within-subject factors (Model 1) to a model in which only absolute difference and EV (Model 2) or only entropy and EV (Model 3) were included as within-subject factors. We modeled the data using the clmm function of the ordinal package (Christensen, 2015) in R (R Core Team, 2013; RRID:SCR_001905). Models included all main effects and contained a full random effects structure (Barr, 2013; Barr et al., 2013). Model comparison was conducted using the anova function in R.
Experiment 2.
For Experiment 2, we investigated whether there was a relationship between OU and/or EV and whether participants were willing to wait to see the outcome of a trial. These data were analyzed using a binomial logistic regression with dependent variable Willingness to wait (Yes/No) and independent variables Participant, OU, and EV. Statistical significance of the model was assessed using χ2, and the amount of variance explained by the model was estimated using Nagelkerke R2. The values for odds ratio (OR) for each independent variable were used to determine the directionality of any significant effects.
Data visualization: behavioral
Experiment 1.
To visualize the behavioral data, the values of OU and EV were divided in percentile bins, such that the 10th percentile represents the 10% lowest values of either OU or EV, the 20th percentile represents the 10%–20% of the lowest values, etc. This was done to enable us to visually compare the effects of OU and EV. To plot the data, we performed the same analysis on the binned data as described above (see Experimental design and statistical analysis: behavioral: Experiment 1). However, to isolate the contributions of OU and EV to curiosity, we added Participant and EV as covariates to visualize the effects of OU on curiosity. The same analysis was performed to visualize the effects of EV on curiosity, but this time Participant and OU were added as covariates. This provided a clear view of the independent effects of OU and EV because even though OU and EV were manipulated independently and showed no linear correlation with each other, they did show a quadratic correlation such that middle values of OU were associated with higher values of EV. These analyses were performed for each participant separately, and the mean curiosity scores for each percentile were calculated by averaging over participants.
Experiment 2.
To visualize the data for the willingness to wait study, the values of OU and EV were divided in percentile bins as described for Experiment 1. To plot the data, we used a Univariate GLM with dependent variable % Willingness to wait, random factor OU, and covariates Participant and EV. The same analysis was performed to visualize the relationship between EV and willingness to wait, but this time EV was added as random factor and Participant and OU as covariates. Again, these analyses were performed for each participant separately, and the mean willingness to wait scores for each percentile were calculated by averaging over participants.
Procedure: fMRI
Experiment 3.
The trial setup for the fMRI study (Experiment 3) was similar to the setup of Experiment 1 (Fig. 1A), except that the timing was adjusted to allow for analyses based on the BOLD response. First, participants saw a screen on which the vase containing the marbles and the points associated with the marbles was presented (4000 ms). Next, participants saw a screen during which they could indicate their curiosity for the outcome on a scale from 1 to 4. This screen was presented for 2500 ms. Then there was a blank screen presented for a jittered duration between 2500 and 4500 ms (uniformly distributed). After the blank screen, participants saw a screen on which the outcome of a trial was either presented or not presented (50% chance), and this screen was displayed for 2000 ms. When the outcome was presented, they saw the colored marble in the middle of the screen and below that the number of points the participant earned. When the outcome was not presented, they saw a black marble in the middle of the screen with question marks at the location where otherwise the number of points would have been presented. Then there was an inter-trial interval consisting of a blank screen, which was presented for a jittered duration between 3500 and 4500 ms (uniformly distributed). After every 9 trials, the duration of the blank screen was prolonged to be jittered between 9500 and 10,500 ms (uniformly distributed). From these prolonged blank screens, a baseline was estimated. Moreover, only 75%–25% and 50%–50% vases were used to reduce differences in visual processing between the different vase configurations. The participants completed a total of 180 trials divided in 4 sessions of 45 trials. After each session, the participants were able to take a short break. In total, the experiment lasted ∼60 min. Also, for this experiment, participants knew that they had no way of influencing whether they would observe the outcome of a trial or not and what that outcome would be. Again, a marble would be selected in every trial, and all the points would be added together and converted to a monetary bonus at the end of the experiment. After functional image acquisition, an anatomical image was acquired.
fMRI acquisition and preprocessing
Functional images were acquired using a multiband imaging sequence (TR = 769 ms, TE = 39 ms, 54 transversal slices, voxel size of 2.4 × 2.4 × 2.4 mm, multiband acceleration factor 6, 52° flip angle). Using the AutoAlign head software by Siemens, we ensured a similar FOV tilt across participants. Anatomical images were acquired using a T1-weigted MP-RAGE sequence, using a GRAPPA acceleration factor of 2 (TR = 2300 ms, TE = 3.03 ms, voxel size 1 × 1 × 1 mm, 192 transversal slices, 8° flip angle).
fMRI data preprocessing was performed using FSL (RRID:SCR_002823) and SPM8 (RRID:SCR_007037). The first seven volumes of each run were discarded to correct for T1 equilibration. The following prestatistics processing was applied; motion correction using MCFLIRT (Jenkinson et al., 2002); non-brain removal using BET (Smith, 2002); spatial smoothing using a Gaussian kernel of FWHM 6.0 mm; and grand-mean intensity normalization of the entire 4D dataset by a single multiplicative factor. Registration to high-resolution structural images was performed using FLIRT (Jenkinson and Smith, 2001; Jenkinson et al., 2002). Registration from high-resolution structural to standard space was then further refined using FNIRT non-linear registration (Andersson et al., 2007a, b). Next, ICA-AROMA (Pruim et al., 2015) was used to reduce motion-induced signal variations in the fMRI data after which the data were high-pass filtered at 1/100 Hz. Finally, data were normalized to standard space and further analyzed using SPM8.
Experimental design and statistical analysis: fMRI
fMRI behavioral analyses.
For the behavioral data of Experiment 3, the same analysis was performed as for Experiment 1 (see Experimental design and statistical analysis: behavioral) using MATLAB (The MathWorks, RRID:SCR_001622) and SPSS (RRID:SCR_002865). Behavioral data visualization for Experiment 3 was done in the same way as for Experiment 1 (see Data visualization: behavioral).
fMRI BOLD analyses.
For each subject, data were modeled using an event-related GLM. The first regressor modeled the period when the vase was presented, serving to induce curiosity in the participants (Curiosity Induction), and had a duration of 4000 ms. The second and third regressor modeled the moment when the outcome of a trial was either presented or not presented and thereby relieved (or not) curiosity in the participants (Curiosity Relief–Yes/Curiosity Relief–No). Both regressors had a duration of 2000 ms. The fourth regressor represented the moment the participants pressed a button to indicate their level of curiosity (Button press) and was modeled as a stick function. The fifth regressor was the baseline (Baseline), which started after every 9 trials when the prolonged blank screen was presented. The duration of this regressor was the duration that the blank screen was presented. All regressors were convolved with a canonical HRF (Friston et al., 1998).
For each trial, values for OU and EV were calculated (see Experimental design and statistical analysis: behavioral) and simultaneously included as parametric modulations of activity during the Curiosity Induction period in the GLM. To this end, the unit height HRF of the Curiosity Induction regressor was convolved with vectors of parametric weights that reflected the trial-by-trial fluctuations of OU and EV. OU and EV were used as parametric modulators at the moment of Curiosity Induction because this was the moment when participants process the vase and when they could make an estimation of the uncertainty of the outcome of a trial (OU) and how much they will approximately earn in that trial (EV). OU was included as the first parametric modulator and EV as the second parametric modulator, such that EV was orthogonalized with respect to OU. The correlation between OU and EV was zero (by design), so the orthogonalization did not affect the EV regressor.
Furthermore, at the moment of trial outcome (Curiosity Relief), we looked at which brain areas respond more strongly to receiving information about the outcome than to not receiving this information and vice versa. We did so by investigating which brain areas were active for the Curiosity Relief contrast (Curiosity Relief–Yes > Curiosity Relief–No) and which brain areas were active for the opposite contrast. We used a primary voxel threshold of p < 0.001 (uncorrected). Inference was based on a cluster-level correction of p < 0.05 (FWE).
To assess residual curiosity-related activity modulations during curiosity induction, after controlling for OU, we modeled the data using another event-related GLM. The GLM was similar to the GLM described above, except that we included an additional parametric modulator Curiosity Rating during the Curiosity Induction period. This parametric modulator reflected the curiosity rating (1–4) that the participants gave in each trial and was included after the parametric modulator for OU. In this way, the parametric modulator for Curiosity Rating was orthogonalized with respect to the parametric modulator for OU and only reflected brain areas that encode residual curiosity variability over and above OU.
ROI analyses.
Additional to the whole-brain fMRI analyses, we defined ROIs based on their significant overall activity modulation during the Curiosity Relief period, using the SPM toolbox MarsBaR (Brett et al., 2002) (RRID:SCR_009605). The rationale for selecting ROIs based on the Curiosity Relief contrast (Curiosity Relief–Yes > Curiosity Relief–No) was that these regions responded more strongly to receiving information about the outcome than to not receiving this information, indicating that they might play a role in processing information about the outcome. We aimed to investigate whether these regions play a role in processing the extent to which participants received an update of information (information prediction error) and reward (RPE). The ROIs based on the opposite contrast (Curiosity Relief–No > Curiosity Relief–Yes) respond stronger to not receiving information about the outcome compared with receiving this information. We aimed to investigate whether these regions play a role in processing the extent to which participants were left with uncertainty about what the outcome would have been (which we refer to as negative information prediction error, see below).
To investigate whether there was a modulation of activity as a function of information and/or RPE effects in these ROIs, we calculated values for the RPE, the information prediction error when the outcome was presented (IPErelief) and the information prediction error when the outcome was not presented (IPEno relief). The RPE (Eq. 3) is the discrepancy between the number of points the participant received (xshown) on a given trial (X) and the number of points the participant expected to receive (EV). The RPE was positive when the participant received more points than expected and negative when the participant received less points than expected. When the outcome was not presented, there was no value for RPE.
In addition to information about the reward, participants get an update of information when the outcome was presented (when they know which marble was selected for them; IPErelief; Eq. 4) as follows:
IPErelief reflects the amount of information that the participant received when being presented with the outcome of a trial. It is calculated by means of the negative natural log of the probability that the shown outcome would be presented P(xshown). This provides a measure of the information content of the outcome given the probabilities. The outcome will lead to a larger information update when the probability that the shown outcome would be selected was lower (see also Shannon, 1948). This value is then multiplied by the absolute difference between the red (x1) and blue (x2) marble points. The absolute difference is a reflection of the variance in points associated with the marbles: the higher this variance, the more uncertain participants were about the outcome and thus the larger the update of information was when participants were presented with the outcome. On the contrary, when the outcome is not presented, participants get no update of information and they remain uncertain about which marble was selected for them (IPEno relief). Therefore, the IPEno relief was calculated as a function of OU. It reflects that the more uncertain participants were at the beginning of a trial, the more uncertainty will remain after not seeing the outcome (Eq. 5). Here, OU(X) reflects the OU of a given trial X (Eq. 1).
The value of IPErelief was always positive and the value of IPEno relief was always negative. In other words: when the outcome was presented, IPE was positive and when the outcome was not presented IPE was negative.
The values of RPE, IPErelief, and IPEno relief were simultaneously included as parametric weights for Curiosity Relief in the GLM. The unit height of Curiosity Relief–Yes was convolved with vectors of parametric weights that reflected the trial-by-trial fluctuations of IPErelief and RPE. IPErelief was included as the first parametric modulator and RPE as the second parametric modulator. As such, RPE is orthogonalized with respect to IPErelief. Because the correlation between IPErelief and RPE is low by design (r = 0.019), this orthogonalization had little influence on the results for RPE. The same was done for Curiosity Relief–No, but then with a vector of parametric weights reflecting the fluctuations of IPEno relief. Again, no parametric weights for RPE could be calculated when no outcome is presented because there is no update about the reward participants received.
For each of the significant ROIs found for the Curiosity Relief contrast, we tested the parameters for RPE, IPErelief, and IPEno relief against 0 using one-sample t tests. The same was done for ROIs that were selected based on the opposite contrast (Curiosity Relief–No > Curiosity Relief–Yes).
Finally, given our specific hypothesis specified in the Introduction, that reward-related areas (e.g., ventral striatum) would be active as a function of EV, RPE, and possibly also as a function of information updating, we investigated mean extracted signal from an a priori defined ROI in the ventral striatum (with MarsBaR; RRID:SCR_009605) (Brett et al., 2002). This ROI was selected from a prior resting state MRI study in which the striatum was parcellated into (anatomically plausible) subregions in a functional data-driven manner, based on intrastriatal functional connectivity analyses that correlated each striatal voxel with all other striatal voxels (Piray et al., 2017). Within the ventral striatum, we investigated the parametric modulation effects of OU, EV, RPE, IPErelief, and IPEno relief. We tested the parameters for these contrasts against 0 using one-sample t tests.
Results
Behavioral results
Experiment 1
The aim of Experiment 1 was to investigate whether and how participants' curiosity was modulated by OU and EV.
Curiosity strongly and monotonically increased with increasing OU (F(1,15) = 103.07, p < 0.001). Conversely, there was no significant modulation of curiosity by EV (F(1,87) = 1.25, p = 0.15; Fig. 2A). There was an interaction between OU and EV (F(1,68) = 1.91, p < 0.001), such that the relationship between OU and curiosity was stronger for medium values of EV, compared with high and low values. This is likely due to the design of the experiment, which had a restricted range of OU values for extreme values of EV, such that extreme values of EV were always associated with low values of OU.
Behavioral results of the three experiments. x axis indicates percentile bins of the values of OU (red) and EV (blue). y axis indicates the mean curiosity rating (A,C) or the percentage willingness to wait (B) for each percentile of OU and EV. In all panels, the effects of EV on curiosity are controlled for OU, and the effects of OU on curiosity are controlled for EV. In Experiment 1 (A), curiosity monotonically increased with increasing OU, but there was no significant modulation of curiosity by EV. This was confirmed by the results of Experiment 2 (B), in which the percentage willingness to wait increased with increasing OU, but not with increasing EV. In Experiment 3 (C), curiosity increased with increasing OU, but also with increasing EV, although the magnitude of the modulation by EV was markedly smaller. For details on behavioral data visualization, see Data visualization: behavioral. Error bars indicate SEM.
We next investigated whether OU predicts curiosity better than entropy or the absolute difference between the marble points alone. The correlation between OU and curiosity was higher (r = 0.698) than the correlation between absolute difference and curiosity (r = 0.523) and the correlation between entropy and curiosity (r = 0.464). This suggests that OU is a better predictor for curiosity than absolute difference or entropy alone, each of which is also strongly correlated with curiosity. To formally assess whether both absolute difference and entropy contributed significantly to curiosity, we compared a model in which both absolute difference and entropy as well as EV were included as within-subject factors (Model 1) with a model in which only absolute difference and EV (Model 2) or entropy and EV (Model 3) were included as within-subject factors. Indeed, we found that Model 1 (Akaike Information Criterion = 19802) explained significantly more variance in the curiosity responses than Model 2 (Akaike Information Criterion = 25702; LR.stat(5) = 5910.1, p < 0.001) or than Model 3 (Akaike Information Criterion = 25899; LR.stat(5) = 6107.2, p < 0.001).
Experiment 2
In Experiment 2, we investigated whether participants would be more willing to wait to see the outcomes of trials they indicated to be more curious about in the first experiment. In other words: were people willing to sacrifice time to satisfy their curiosity?
The results show that people were more willing to wait (χ2(3) = 2944.29, p < 0.001, Nagelkerke pseudo R2 = 0.505) when OU was higher (OR = 1.11, p < 0.001), but that willingness to wait was not modulated by EV (OR = 1.00, p = 0.98). These results are consistent with Experiment 1, showing that people not only indicated to be more curious when OU is higher, but that they were also more willing to wait to see this outcome (Fig. 2B). Similarly, people were not more willing to wait for trials with high compared with low EV, supporting the findings of Experiment 1. In Experiment 1, there was a significant interaction effect between OU and EV on curiosity. Therefore, we investigated whether adding the interaction in this model would improve the model fit. However, the model did not get better by adding the interaction between OU and EV (χ2(4) = 2.48, p = 0.12). This indicates that the effects of OU on willingness to wait did not differ between different values of EV.
Experiment 3
As in Experiment 1, curiosity increased with increasing OU (F(1,14) = 69.87, p < 0.001), replicating the finding of Experiment 1. In Experiment 3, however, curiosity also increased with increasing EV (F(1,30) = 3.26, p < 0.001; Fig. 2C), although the magnitude of the modulation was markedly smaller. There was also an interaction between OU and EV (F(1,61) = 3.00, p < 0.001), such that the relationship between OU and curiosity was stronger for medium values of EV, compared with high and low values.
Neuroimaging results
Curiosity induction
First, we examined activity when curiosity was induced (during vase presentation) as a function of OU and EV. There was one cluster that showed a significant increase in activity with increasing OU, located in the left inferior parietal lobule (Table 1; Fig. 3). Somewhat surprisingly, there was a large network of regions that showed a significant increase in activity with decreasing OU (Table 1; Fig. 3), including the hippocampus, precuneus, and several clusters within the temporal and frontal lobe. No significant positive modulation of EV was observed, whereas activity in the calcarine sulcus increased as a function of decreased EV (Table 1). When examining residual curiosity-related activity during curiosity induction, after controlling for OU, there was one cluster that showed a significant increase in activity with increasing curiosity rating, located in the left pre-supplementary motor area (Table 1). Finally, ROI analyses in the ventral striatum showed no significant parametric modulation effects of OU (t(23) = −1.44, p = 0.16) and EV (t(23) = −1.75, p = 0.093) at the time of curiosity induction.
Brain regions associated with OU and EV during curiosity inductiona
Clusters showing a positive (red) or negative (blue) activity as a function of OU. The maps are dual-coded and simultaneously display the contrast estimate (x axis) and unthresholded t values for the curiosity relief contrast (y axis). Thereby, the hue indicates the size of the contrast estimate, and the opacity indicates the height of the t value. Significant clusters (cluster-level corrected, FWE, p < 0.05) are encircled in white. One cluster in the left inferior parietal lobule showed a significant increase in activity with increasing OU, whereas a large network of regions showed a significant increase in activity with decreasing OU. The z coordinates correspond to the standard MNI brain. Neuroimaging data are plotted using a procedure introduced by Allen et al. (2012) and implemented by Zandbelt (2017).
Curiosity relief
Next, we looked at the moment when participants' curiosity was either relieved or not (i.e., when they were presented with the outcome or not). Three brain regions showed larger response when curiosity was relieved (Curiosity Relief–Yes > Curiosity Relief–No): the right insula, the right middle orbitofrontal cortex, and the right inferior parietal lobule (Table 2; Fig. 4). When the outcome was withheld (Curiosity Relief–No > Curiosity Relief–Yes), this led to larger response in the right middle occipital gyrus, the left frontal superior gyrus, and the right frontal middle gyrus (Table 2; Fig. 4).
Brain regions associated with curiosity reliefa
Clusters showing a positive (red) or negative (blue) activity when comparing relieved versus non-relieved curiosity, at the time of the outcome presentation. Three brain regions showed larger response when curiosity was relieved (Curiosity Relief–Yes > Curiosity Relief–No): the right insula, the right middle orbitofrontal cortex, and the right inferior parietal lobule. When the outcome was withheld (Curiosity Relief–No > Curiosity Relief–Yes), this led to larger response in the right middle occipital gyrus, the left frontal superior gyrus, and the right frontal middle gyrus. Other conventions are as in Figure 3.
We next investigated whether activity for nodes in these networks was linearly modulated by the size of the information update (i.e., IPErelief and IPEno relief). Within the nodes that were activated when the outcome was presented and curiosity was relieved, the ROI analyses showed a linearly increasing response as a function of IPErelief magnitude in the right insula (t(23) = 2.70, p = 0.013). This indicates that this region was involved in processing the prediction error of information when participants observed the outcome of a trial. No significant parametric modulation effects for IPErelief were found in the right middle orbitofrontal cortex region (t(23) = −1.30, p = 0.21) or in the inferior parietal lobule region (t(23) = −0.59, p = 0.56). For the nodes that were active when the outcome was withheld, there was a significant parametric modulation effect of negative IPE (IPEno relief) in the right middle occipital gyrus (t(23) = −2.91, p = 0.0079), such that this area showed a higher activity on trials where participants were most uncertain about the outcome (highest OU, resulting in the largest negative IPE). The left frontal superior gyrus (t(23) = 0.96, p = 0.35) and the right frontal middle gyrus (t(23) = 0.91, p = 0.37) were not modulated by IPEno relief.
To summarize, the ROI analyses showed that the right insula was more active for higher positive information prediction errors (IPErelief), whereas the right middle occipital gyrus was more active for higher negative information prediction errors (IPEno relief). On the contrary, there was no significant parametric modulation effect of IPEno relief in the right insula (t(23) = −1.30, p = 0.21) and for IPErelief in the right middle occipital gyrus (t(23) = −1.66, p = 0.11). This indicates that the insula solely responded to prediction errors of information when the outcome was presented to the participants, such that this region was more active when they received a larger update of information. Likewise, the right middle occipital region solely responded to prediction errors of information when the outcome was not presented, such that this region was more active when uncertainty was highest, but curiosity was not relieved.
No significant parametric modulation of BOLD activity due to RPE (t(23) = 1.48, p = 0.15), IPErelief (t(23) = −0.94, p = 0.36), or IPEno relief (t(23) = −1.66, p = 0.11) was found in the ventral striatum at the time of curiosity relief.
Discussion
Here, we examined the cognitive and neural constituents of curiosity. We found that curiosity and willingness to wait increased with increasing uncertainty about a lottery outcome. Curiosity induction through OU was related to increased BOLD activity in the inferior parietal lobule, whereas curiosity relief elicited increased BOLD activity in the insula, orbitofrontal cortex, and parietal cortex. Importantly, outcome-related signal in the insula increased linearly with increasing information prediction error provided by the outcome. Conversely, curiosity was not strongly related to the EV of lotteries.
Curiosity is modulated by OU
Experiments 1 and 2 revealed that, even though information did not help participants to perform better or to maximize rewards, individuals were more curious about outcomes that provided them with a larger information update (i.e., when OU was higher). Experiment 1 goes beyond studies on curiosity that indicated that humans and animals seek information to reduce their uncertainty (Berlyne, 1962; Lieberman et al., 1997; Bromberg-Martin and Hikosaka, 2009, 2011) by showing that, even in a passive observation task, humans show a drive towards gaining information and reducing uncertainty about what will happen.
One might argue that the findings of Experiment 1 reflect participants' compliance with task demands (i.e., showing higher curiosity about uncertain outcomes because they felt like this was expected of them). To minimize such demand characteristics, we performed Experiment 2 in which curiosity was probed implicitly by means of participants' willingness to wait for the outcome. We found that participants' willingness to wait increased with increasing OU, indicating that individuals are willing to “pay” with time for information to satisfy their curiosity. This supports the findings of Experiment 1 and makes it less likely that demand characteristics explain our results. Critically, this suggests that a preference for information is present even when receiving information is costly (Roper et al., 1999) and rewards have to be sacrificed (Stagner et al., 2010; Blanchard et al., 2015). This indicates that information gain may be intrinsically valuable and strengthens the hypothesis that it is evolutionarily adaptive to reduce our uncertainty about the world (Loewenstein, 1994; Gottlieb et al., 2013).
Unlike the present study, prior work on human curiosity using trivia questions (e.g., Gruber et al., 2014; Marvin and Shohamy, 2016) suggested that individuals are mostly curious about information of intermediate uncertainty (Kang et al., 2009). Individuals appeared to be mostly curious about questions about which they exhibit intermediate levels of confidence compared with questions about which they were very confident or not confident at all. This contrasts with our observation of a linear relationship between uncertainty and curiosity. However, it might reflect that participants in previous work were not curious about questions with highest uncertainty (i.e., lowest confidence) because they were simply not interested in the topics of these questions. In other words, there might be a correlation between one's knowledge and one's curiosity about the subject, confounding the relationship between curiosity and uncertainty. Here, this confound was avoided by experimentally manipulating OU in a quantitative and controlled manner.
Somewhat surprisingly, we found no effect of EV on curiosity. Perhaps our reward manipulation was not effective and rewards did not yield sufficient interest because individuals could not maximize rewards or there was no opportunity to make an explicit choice. Alternatively, or additionally, the EV computation might have been too demanding in the behavioral experiments. In Experiment 3, we used only two vase configurations, which made it easier to encode both the OU and the EV. Indeed, in Experiment 3, we found a significant increase in curiosity with increasing EV. Nevertheless, this relationship was markedly smaller than the relationship between OU and curiosity, strengthening the finding that uncertainty is a strong inducer of curiosity, beyond reward.
Curiosity induction elicits parietal signal, whereas curiosity relief elicits frontal signal
Investigation of the BOLD signal during curiosity induction revealed a usual suspect as a function of OU, namely, the parietal cortex (Huettel et al., 2005). This finding is unsurprising because prior work has implicated this region in processing OU. The key novel finding is that BOLD signal in the insula covaried with the information prediction error at the moment of curiosity relief, such that insular activity linearly increased with the amount of information participants gained by seeing the outcome (information prediction error). Intriguingly, insular signals have previously been observed to vary with risk prediction errors, which represent the degree to which outcomes are surprising in a gambling task (Preuschoff et al., 2008). This risk prediction error signal was interpreted to reflect a role for the insula in risk prediction learning, relevant for learning when to avoid or approach risk. While risk and information prediction errors are both measures of the degree to which outcomes are surprising in an uncertain environment, a key difference is that the concept of risk is tightly linked to choice, whereas information prediction errors can be elicited in a passive observation task, as in the current study. As such, the current study suggests that the insula is not just involved in learning about risk to optimize choice, but may also contribute to knowledge acquisition by signaling information prediction error more generally. For future studies, it might be interesting to investigate whether information prediction errors found for non-instrumental curiosity have consequences for learning, memory, and hippocampal activity found with trivia questions (cf. Gruber et al., 2014; Marvin and Shohamy, 2016).
In addition to these insular signals, we found overall increased BOLD signal in the right orbitofrontal cortex and the right inferior parietal lobule at the time of curiosity relief. These findings generalize prior findings that the insula and orbitofrontal cortex respond to perceptual curiosity relief (Jepma et al., 2012) to nonperceptual curiosity relief, suggesting that these areas might be involved more generally in gaining information. The insula might do so by computing specific information prediction error signals, which help us to optimize our current world model. The orbitofrontal cortex, however, has been argued to signal information about reward, reward-learning, and reward-related cognition (Wallis, 2007; Padoa-Schioppa, 2011; Rushworth et al., 2011; Wilson et al., 2014; Blanchard et al., 2015). Given this established link between orbitofrontal cortex and reward, it is unsurprising that this region was more active when participants saw the outcome than when they did not.
Perhaps surprisingly, we found no activity modulation by reward or information prediction error in the ventral striatum. While this may partly reflect the lower signal-to-noise ratio in deep brain structures due to the specific fMRI sequence we used (see Materials and Methods), we think the most relevant factor here could be the use of a passive observation task. Unlike choice tasks, participants had no means to influence their task performance or improve the amount of received rewards. Therefore, participants may be mainly driven by OU, instead of reward, and this might be reflected in the absence of activity modulations in reward-related areas, such as the ventral striatum.
As mentioned above, we found that BOLD signal in the left inferior parietal lobule increased with increasing OU at the time of curiosity induction. This extends previous findings by Huettel et al. (2005), who found increased activity in parietal cortex with increasing OU in a decision-making task. Given our finding that the inferior parietal lobule is similarly involved in processing OU in a passive observation task, it could be that this area is more generally involved in processing OU, which may in turn contribute to curiosity induction. This might be due to requirement of more attentive control in situations of high OU. Possibly, (curiosity-driven) actions associated with acquiring new information might require more attentive control, whereas actions with low uncertainty are more habitual and require little attention (Shenhav et al., 2013; Fan, 2014). Perhaps, these curiosity-inducing signals in parietal cortex operate independently from signals responsible for signaling expected rewards, as previous work in macaque monkeys has indicated that parietal neurons may encode the expected reduction in uncertainty an action will bring, independent of expected reward and RPEs (Foley et al., 2017). Finally, although a large portion of the variability in curiosity was accounted for by OU, there was also some residual variability in curiosity ratings not accounted for by OU (or EV). Supplementary analyses revealed such residual (not OU-related) curiosity signals in the left pre-supplementary motor area, a region commonly implicated in monitoring actions (Rushworth et al., 2007), and the value of information in a decision-making context (Behrens et al., 2007).
In conclusion, our study sheds light on the cognitive and neural constituents of curiosity by demonstrating that OU drives curiosity strongly. This curiosity induction is accompanied by BOLD signal in parietal cortex, whereas its release is accompanied by BOLD increases in the insula, orbitofrontal cortex, and parietal cortex. Most strikingly, insular activity increased linearly with increasing information update the outcome provided. Together, these findings point to a fundamentally adaptive role of curiosity as a drive to improve one's current world model and implicate the parietal cortex and insula in this fundamental trait.
Footnotes
This work was supported by The Netherlands Organisation for Scientific Research NWO Vidi Award 452-13-016 to F.P.d.L. and NWO Vici Award 453-14-015 to R.C., James McDonell Foundation JSMF Scholar Award 220020328 to R.C., and the EC Horizon 2020 Program ERC Starting Grant 678286 to F.P.d.L. We thank David Richter and Monja I. Froböse for support in data analysis; and Eliana Vassena for useful comments on an earlier version of the manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to Lieke L. F. van Lieshout, Donders Institute for Brain, Cognition and Behaviour, Radboud University, P.O. Box 9101, 6500 HB Nijmegen, The Netherlands. l.vanlieshout{at}donders.ru.nl
