Abstract
Resting-state fMRI (rsfMRI) reveals brain dynamics in a task-unconstrained environment as subjects let their minds wander freely. Consequently, resting subjects navigate a rich space of cognitive and perceptual states (i.e., ongoing experience). How this ongoing experience shapes rsfMRI summary metrics (e.g., functional connectivity) is unknown, yet likely to contribute uniquely to within- and between-subject differences. Here we argue that understanding the role of ongoing experience in rsfMRI requires access to standardized, temporally resolved, scientifically validated first-person descriptions of those experiences. We suggest best practices for obtaining those descriptions via introspective methods appropriately adapted for use in fMRI research. We conclude with a set of guidelines for fusing these two data types to answer pressing questions about the etiology of rsfMRI.
Introduction
Resting-state fMRI (rsfMRI) refers to fMRI scans of subjects instructed to let their mind wander (i.e., think about whatever comes to mind) and fixate on a crosshair (or with eyes closed). No other motor, perceptual, or cognitive processes are explicitly demanded. This experimental simplicity makes rsfMRI data easy to collect and share, amenable for interspecies comparisons, and ideal for biomarker development. Unfortunately, the simplicity of rsfMRI also makes determining etiology of signals (e.g., neuronal, physiological, or artifactual) and establishing putative relationships to ongoing mental processes quite challenging. Nonrest imaging modalities can rely on task timing and cognitive constraints, as well as subjects' responses, to isolate neuronally driven fluctuations and interpret their cognitive correlates. Unfortunately, rsfMRI cannot resort to similar procedures because rsfMRI scans are rarely annotated with sufficiently detailed information about the cognitive, perceptual, and motor processes that unfold in subjects' minds and bodies during resting-state scans (i.e., a subject's ongoing experience during rest).
Examination of neuronal contributions to rsfMRI has been the focus of ample research (Box 1). Collectively, this research has cemented the view that low-frequency fluctuations in rsfMRI are primarily of neuronal origin and reflect meaningful aspects of the brain's functional organization. However, this “neurocentric” model is inconclusive (Lu et al., 2019), and effectively isolating and interpreting neuronally induced fluctuations remains challenging. In addition to the well-documented confounding effects of excessive head motion (Power et al., 2018), investigators have also demonstrated that network-like structures can arise in rsfMRI data from physiological phenomena: for example, voxelwise differences in vascular responses to respiration and cardiac events (Chen et al., 2020), and spread of other non-neural systemic oscillations1 of unknown origin through cerebral vasculature (Tong et al., 2015). Accordingly, basic network features, such as confinement to gray matter, interhemispheric symmetry, and overlap with functionally coherent cortical systems, although promising, become insufficient to claim neuronal origin. Additional evidence is needed to minimize interpretational uncertainty. Although desirable properties, such as echo-time dependence2 (Kundu et al., 2012) and conscious state dependence (Barttfeld et al., 2015), are sometimes explored, additional ways to substantiate interpretation are quite limited when concurrent electrophysiological recordings do not exist. Ideally, establishing how ongoing experience can be expected to regulate functional connectivity at regional and network levels could appreciably strengthen our confidence about how to interpret rsfMRI results. Unfortunately, as the role of ongoing experience in rsfMRI remains largely unexplored, little progress has been made on this front.
Neurocentric model of rsfMRI
In 1995, Barat Biswal and colleagues reported that low-frequency fluctuations in fMRI signals obtained from awake resting human subjects showed patterns of spatial correlation suggestive of a neurophysiological origin (Biswal et al., 1995). While their study focused on motor cortex, soon after other groups reported similar results for visual cortex (Hampson et al., 2004), auditory cortex (Lowe et al., 1997), language regions (Hampson et al., 2002), and eventually the default mode network (Greicius et al., 2003). In all these seminal studies, researchers took extreme precautions to ensure artifactual sources (e.g., system noise, physiological noise, head motion) could not account for the observed patterns. Still, evidence was not sufficiently conclusive to ensure the observed phenomena were purely neuronal.
In the 25 years that followed (for an excellent review of the first 15 years, see Lowe, 2010), dozens of studies aimed to answer this critical question: “To what extent is resting-state functional connectivity driven by underlying neuronal phenomena?” The diversity of approaches that researchers have adopted to test this idea is astonishing. For example, some have confirmed signal characteristics that fit expectations of a neuronally induced, BOLD-like phenomenon, including confinement to gray matter (Biswal et al., 1995), exponential dependence with echo time (Peltier and Noll, 2002), attenuation under hypercapnia (Biswal et al., 1997a), and manifestation in cerebral blood flow measures (Biswal et al., 1997b). Others have evaluated the effects of structural insults and found that compromised anatomic connectivity, such as in callosal agenesis (Lowe et al., 1997) and multiple sclerosis (Lowe et al., 2002), results in weakened or absent functional connectivity. In parallel, many have reported significant relationships between resting functional connectivity patterns and clinical symptoms (for review, see Lee et al., 2013), as well as with individual variability in behavioral traits, such as fluid intelligence (Finn et al., 2015), impulsivity (Li et al., 2013), neuroticism, and extraversion (Hsu et al., 2018). Another important line of research demonstrates that resting-state functional connectivity patterns mimic well-established principles of functional brain organization both at the whole-brain level (Smith et al., 2009), and at the finer scale of functional subdivisions in complex structures, such as the striatum (Di Martino et al., 2008), the anterior cingulate cortex (Margulies et al., 2007), and the thalamus (Zhang et al., 2008). Yet, perhaps the strongest evidence supporting a neuronal basis for resting-state functional connectivity comes from studies looking at concurrent hemodynamic and electrophysiological recordings. Key findings in this line of research include strong agreement between spatial distribution of low-frequency fluctuations and that of the following: (1) the amplitude of local field potentials in the γ band in anesthetized monkeys (Shmuel and Leopold, 2008), (2) neural firing in awake resting mice (Ma et al., 2016), (3) electrocortical recordings in anesthetized patients with epilepsy (Nir et al., 2008), and (4) electroencephalographic recordings in awake resting healthy humans (Laufs et al., 2003).
Here, we argue that resting subjects engage in rich mental experiences that correspond to bursts of short, spatially distributed activity, which in turn drive rsfMRI functional connectivity estimates. Moreover, we propose measuring these ongoing experiences with introspection to empirically test this hypothesis.
Resting ongoing experiences are rich and individually diverse
The first pillar to ensure steady progress in advancing our conception of the joint dynamics of ongoing experience and rsfMRI data are to explicitly acknowledge that resting state is not a single state of mind, but a succession of cognitive, emotional, perceptual, and motor processes, both conscious and nonconscious, unique to each scan (Fig. 1A). In other words, researchers should not assume that neural dynamics in the absence of experimentally imposed sensory stimulation or cognitive demands lack correspondence to meaningful experiences. Scanners hardly resemble sensory-deprived environments, and subjects' minds are unlikely idle. First, scanners generate loud pulsed acoustic noises during data acquisition, which can affect functional connectivity (Dionisio-Parra et al., 2020). Second, scanner room lighting conditions are often sufficiently lit for subjects to appreciate and direct their attention to different areas within their restricted field of view. Third, in addition to moving their heads, subjects often reposition themselves and voluntarily move their limbs (Tan et al., 2017). Fourth, subjects often report periods of heightened sensation (Delamillieure et al., 2010), including itches, physiological urges (e.g., urination, thirst), and muscular discomfort. Fifth, subjects engage in multiple spontaneous and goal-oriented cognitive processes (Diaz et al., 2013, 2014). Sixth, subjects' vigilance level fluctuates as scanning progresses (Tagliazucchi and Laufs, 2014). Despite the richness and diversity of cognitive, perceptual, and motor processes composing subjects' ongoing experience during rest, researchers rarely acknowledge these contributions, and instead use a single blanket term, such as “intrinsic” or “spontaneous,” to describe all neuronal activity observed during rsfMRI (Kucyi et al., 2018). Such an overly simplistic characterization reflects our limited understanding of how idiosyncratic ongoing experiences during resting state may shape functional connectivity. More importantly, it neglects converging evidence suggestive of a significant role for aspects of ongoing experience: for example, self-initiated motion (Petridou et al., 2013; Winder et al., 2017), content (Gorgolewski et al., 2014; Gonzalez-Castillo et al., 2019), form3 (Gorgolewski et al., 2014), spontaneity4 (Wang et al., 2018b), and temporal focus (Vatansever et al., 2020) of ongoing thoughts), other than shift in vigilance (Tagliazucchi and Laufs, 2014; Laumann et al., 2017), in modulating patterns of rsfMRI functional connectivity.
Schematic representation of the main premise of this work. A, Subjects have rich ongoing experiences characterized by a succession of cognitive, emotional, and perceptual processes unique to each scan. B, Those experiences manifest in rsfMRI data in the form of short bursts of distributed neuronal activity (i.e., coactivation events). C, Coactivation events, in turn, drive estimates of functional connectivity.
Moment-to-moment fluctuations in rsfMRI
The second pillar of the present argument is that a sound understanding of rsfMRI will emerge from exploration of its most dynamical aspects (Fig. 1B,C). For almost two decades, rsfMRI research focused almost exclusively on effects estimated using entire time series based on the assumption that functional connectivity remained constant for the entirety of individual scans. However, recent research demonstrates that rsfMRI connectivity is time-varying (for review, see Preti et al., 2017; Lurie et al., 2020); and functional networks can undergo several reconfigurations during individual rsfMRI scan, which often takes 5-15 min (Chang and Glover, 2010; Gonzalez-Castillo et al., 2014). Importantly, detailed explorations of moment-to-moment fluctuations in rsfMRI data have demonstrated that static functional connectivity estimates (e.g., intrinsic networks) are driven primarily by a small number of brief, strong, temporally sparse, spatially distributed bursts of activity, known as coactivation events (Fig. 1B) (Tagliazucchi et al., 2011, 2016; Petridou et al., 2013; Allan et al., 2015; Gutierrez-Barragan et al., 2019). Although these coactivation events only make up <10% of the data, they suffice to faithfully reproduce functional connectivity patterns estimated using full-time series in healthy individuals (Tagliazucchi et al., 2016; Gutierrez-Barragan et al., 2019), and to reproduce functional connectivity disruptions associated with sleep (Tagliazucchi et al., 2016) and disease (Tagliazucchi et al., 2011). Importantly, these findings cannot be reproduced with a similar number of randomly selected samples, suggesting that coactivation events are important drivers of functional connectivity in rsfMRI data. Mapping the ongoing experiences accompanying these important neuronal activations could hold the key to interpreting rsfMRI functional connectivity.
Four additional observations emphasize the prominent role of coactivation events in rsfMRI data. First, coactivation events follow the temporal signature of neuronally induced hemodynamic events; and as such, they can be reliably detected using hemodynamic deconvolution (Tagliazucchi et al., 2011; Allan et al., 2015). Second, concurrent hemodynamic and electrophysiological recordings demonstrate strong correspondence between hemodynamic and neuronal coactivation events (Matsui et al., 2019). Third, removal of coactivation events from rsfMRI data (via regression or scrubbing) significantly reduces the detectability of intrinsic functional connectivity networks (Allan et al., 2015). Fourth, coactivation events have been linked to self-initiated behavior (Petridou et al., 2013; Allan et al., 2015; Gonzalez-Castillo et al., 2019). Jointly, these observations not only indicate that rsfMRI coactivation events are neuronally driven, they also reveal that coactivation events are linked to observable behaviors: for example, human motion (Petridou et al., 2013) and mice whisking (Winder et al., 2017). However, the degree to which the relationship between coactivation events and behavior can be extended to mentation-related aspects of resting ongoing experiences remains unknown.
Given the above-mentioned role of coactivation events in shaping rsfMRI functional connectivity, their demonstrated neuronal origin, and their link to observable behaviors, we postulate that establishing the relationship between coactivation and mentation events is the most effective way to explore the cognitive correlates of rsfMRI. Importantly, such a scientific endeavor will require access to the timing, content, and modality (e.g., visual, auditory) of covert cognition unfolding during rest. Arguably, some of this information can be obtained via traditional, concurrent, nondisruptive third-person measures, such as those provided by skin conductance (e.g., to detect periods of elevated emotional valence) and eye tracking (e.g., to detect shifts in visuospatial attention). However, a detailed portrayal of ongoing mentation can only be achieved by reports from the first-person perspective as provided by introspection. We next briefly introduce the scientific methodology of introspection with an emphasis on its application to neuroimaging research.
Introspection: a window into the mind
Historically, introspection denotes “attention on the part of an individual to his own mental states and processes, as they occur, with a view of knowing more about them” (Baldwin, 1901). Here, we use this term differently to designate a set of scientific instruments by which experimenters gain access into one's ongoing experience. Over the years, introspection methods have consistently been met with strong skepticism over their reliance on subjective reports (Nisbett and Wilson, 1977; Skinner, 1977) (for our rebuttal, see Box 2). A critical issue for scientifically valid introspection is the distinction between content and interpretation. Just as scientists minimize bias by withholding theoretical interpretations when collecting and analyzing data, only introspective methods directed at gaining insights into the contents of our experience and not about the reasons for that experience (i.e., interpretation) can meet scientific criteria. Indeed, despite its critics, content-oriented introspection has played an important role in shaping the early years of experimental psychology (Box 3), and continues to be a vital instrument (Box 2) in cognitive neuroscience: for example, pain (Ploner et al., 2010), personality (Eisenberger et al., 2005), emotions (Northoff and Heinzel, 2006), dreaming (Horikawa et al., 2013), states of consciousness (Schooler, 2002), mind wandering (Martinon et al., 2019), and psychiatric diagnosis (Diagnostic and statistical manual of mental disorders, Ed 5, American Psychiatric Association DSM-5 Task Force, 2013; and International classification of diseases, Ed 11, World Health Organization, 2018).
Addressing subjectivity in introspection research
Many researchers object to measurements based on introspection, largely because they assume the subjective nature of such data is somehow contradictory to the scientific pursuit of objective truth. However, subjective reports are widely used in epidemiology, psychology, and other social sciences. These fields have developed extensive theory and methodology for establishing the necessary reliability, validity, and generalizability for any given psychometric instrument, just as is done for physical measurement systems (Kimberlin and Winterstein, 2008). Furthermore, the explanatory power of subjective reports is evident in their central role in several of the biggest success stories in neuroscience.
The clearest example is the field of psychophysics, which can be traced directly back to Wundt's pioneering use of introspection (Wundt and Wirth, 1902). Psychophysics investigates mechanisms of sensation and perception by obtaining subjective ratings of perceived stimulus intensity in response to precisely controlled presentation of physical stimuli. Converging evidence across numerous subjective measurement techniques established reliable and precise power law relationships between physical and perceived stimulus intensity for vision, audition, taste, and pain (Price et al., 2002). Importantly, these perceptual response curves were found to be directly proportional to the neural firing rates in afferent sensory nerves (Borg et al., 1967), and similar mappings between subjective ratings and activity in cortical and subcortical regions have been used to explain other phenomena in perception (e.g., multisensory integration) (Stein and Meredith, 1993). Notably, psychophysics methods have been adapted to cases in which experimenters cannot control physical stimuli by carefully operationalizing independent and dependent variables, ensuring participants understand those definitions, and obtaining ratings using well-validated scales (Price and Barrell, 2012).
Neuroeconomics also embraces subjectivity to improve our understanding of reward learning and decision-making. Behavioral economics applies psychology research on subjective biases (e.g., heuristics, loss aversion) to help explain irrational choice behavior. Neuroeconomics builds on this work by investigating the neural underpinnings of these value-based decisions by comparing subjective value ratings to neural recordings, which has revealed single neurons (Platt and Glimcher, 1999; Schultz, 2010) and cortical areas (Schultz, 2010) that encode value signals. These findings represent substantial progress toward one of the goals of neuroeconomics: redefining subjective value as objectively measured spikes per second in reward circuitry (Glimcher, 2009).
Combining measurements of subjective experience and neural activity has also driven advances in research on meta-cognition (Fleming and Dolan, 2012), placebo effects (Vase et al., 2011), and consciousness (Block, 2019), among others. Collectively, these diverse examples clearly demonstrate that first-person data are scientifically valid, widely used, complimentary to third-person data, and synergistic with neural recordings in the investigation of neural mechanisms of ongoing experience.
History of introspection in psychology
The influential work of several psychologists from the late 1800s to early 1900s demonstrates the utility of introspection as a scientific method in experimental psychology. Considered the father of introspection in psychology, Wilhelm Wundt adopted a highly controlled and systematic approach to investigating mental events (Wundt, 1907). He experimentally controlled stimulus presentation to elicit consistent and replicable percepts. Wundt referred to this approach as internal perception, which aimed to approximate reliability of data obtained in the natural sciences. Wundt also trained his observers to ensure that their passive observation happened quickly and accurately (Wundt, 1907). For instance, observers were systematically trained to report whether a tone is perceived as higher or lower in pitch compared with the last tone in a series of repeated presentations. The strict experimental conditions imposed by Wundt naturally limited his research to sensations and perceptions. Although Wundt's work fell short of providing a means to gain a comprehensive understanding of our inner experience, it highlighted the importance of training an observer to become experienced in introspection.
Another major proponent of introspection was Edward Titchener, a student of Wundt who also believed that accurate introspection was attainable by training (Titchener, 1912). In addition to internal perception, Titchener used retrospection in which the observers first have the experience, unattended and thus untainted, then recall the experience immediately after it occurs from memory. Although susceptible to memory distortion, retrospection was considered the only way in which the study of one's conscious experience is not interrupted by the act of actively observing one's experience (Boring, 1953; Danzinger, 1980). Titchener's work underscores the value of immediate retrospection in revealing an untainted inner experience of the observer.
In contrast to the experimental approach of Wundt, his contemporary, William James, investigated the stream of consciousness using qualitative retrospection. This approach involved direct, in-depth investigation of one's subjective experience beyond sensations and perceptions (James et al., 1981). James believed that introspection required retrospection, as this was the only way to not interfere with the actual experience itself. He did not conduct tightly controlled experiments to avoid reducing the stream of consciousness to sensory percepts. Rather, he was in favor of a more qualitative approach to examine the entirety of our conscious experience. Despite the qualitative nature of James' approach, his work set the foundation for the development of the most influential findings about human consciousness in psychology, attesting to the value that qualitative methodologies add to our understanding of the conscious experience.
Notably, introspection has also contributed to our overall understanding of ongoing experience in rsfMRI. Using self-report questionnaires, researchers have uncovered key dimensions of ongoing experience during rest (e.g., discontinuity of mind,5 theory of mind,6 planning) (Diaz et al., 2013, 2014; Gorgolewski et al., 2014), described considerable levels of intersubject variability in resting-state ongoing experiences (Delamillieure et al., 2010), identified patterns of thought (e.g., future planning, current concerns) that can explain interindividual differences in static resting-state functional connectivity (Vatansever et al., 2020), and established statistical relationships between the temporal persistence (dwell time) of certain functional connectivity configurations (brain states) and overall patterns of thinking (e.g., future planning) (Karapanagiotidis et al., 2020). To prevent disruption of the ongoing experience as it unfolds (i.e., observer effects), all those aforementioned studies required subjects to retrospectively (with hindsight) reflect and report summary characteristics of their experience immediately after scanning concluded (Fig. 2A). Although such reports can help establish statistical relationships between summary characteristics of ongoing experience and neuroimaging data (trait-level links), they lack temporal precision to reveal “one-to-one” links between individual instances of ongoing mentation and contemporaneous coactivation events (instance-level links). In addition, retrospection inherently relies on episodic memory, rendering it susceptible to distortion during encoding, storage, or recall (Lapping-Carr and Heavey, 2017). Finally, retrospective methods aggregate information across long epochs of time, blurring details in the content, modality, and quality of specific episodes of the resting ongoing experience occurring at any given instant (i.e., temporal blurring effects). In other words, although retrospective methods may play a key role when exploring the etiology of functional connectivity during resting state, they may not suffice when trying to explore the proposed link between specific coactivation events and aspects of ongoing experience. For this particular goal, retrospective approaches must be complemented with additional methods that can provide time-resolved information about ongoing experience (Fig. 2B). We discuss those next.
A, Retrospective introspection relies on thought probes/surveys that take place at the conclusion of a scan and gather information about the overall ongoing experience of subjects during the entirety of the resting-state scan. B, In-the-moment introspection presents thought probes at pseudo-random times during the resting-state scan. Each of those probes is aimed at obtaining a characterization of the ongoing experience unfolding right before the probe was presented (e.g., it gathers information about a short, i.e., few seconds, period of time).
“In-the-moment” introspection
One method able to meet the requirement of time-resolved information is thought sampling (also known as experience sampling) (Martinon et al., 2019). This technique presents thought probes, namely, brief sets of force-choice or open-ended questions, at pseudorandom intervals that subject answer based on the characteristics of their ongoing experience at the moment immediately before the onset of the probe (e.g., Smallwood and Schooler, 2006). This “in-the-moment” introspection technique is common in mind wandering7 research (for review, see Martinon et al., 2019) a field in cognitive neuroscience with strong conceptual links to rsfMRI (Box 4) that has traditionally explored one's ongoing experience when attention is directed away from an external task (Smallwood and Schooler, 2006). Given that the depiction of mind wandering often involves gathering knowledge about the focus of attention and content of the wandering thoughts, introspection has always played an important role in mind-wandering research. Indeed, although it is possible to infer from behavioral performance that one's attention has wandered away from an experimental task, it is impossible to know where attention has drifted. Introspection provides the most direct measure of where the ongoing experience has wandered off.
Mind wandering and its relation to rsfMRI
The fields of mind wandering and rsfMRI present seemingly contrasting definitions of mind wandering. In mind-wandering studies, mind wandering is traditionally defined as attention away from an ongoing, externally oriented task (Christoff et al., 2009; Kam et al., 2011, 2012, 2013). In rsfMRI studies however, which generally instruct subjects to let their minds wander in a task-free environment, mind wandering simply implies that subjects think about whatever comes to mind. Indeed, the task here is to “mind wander.”
Despite such nuanced differences in procedural definitions, a recent shift in the mind-wandering field (away from the singular task-oriented definition) and the rsfMRI field (away from static metrics aggregating across entire time series) reveals a common interest in understanding the content and dynamics of ongoing experience. In particular, increased efforts in mind-wandering research to explore the content and dynamics of wandering thoughts (Christoff et al., 2016; Zanesco, 2020) are paralleled by a growing interest in rsfMRI research to understand the joint dynamics of signal fluctuations and covert cognition (Gonzalez-Castillo et al., 2019) (see Moment-to-Moment fluctuations in rsfMRI). This shared interest has converged on similar patterns of findings. For instance, both fields have independently established clinical correlates: abnormal patterns of mind wandering and rsfMRI have been identified in depression (Greicius, 2008) and Alzheimer's disease (Greicius, 2008; O'Callaghan et al., 2019).
The methodological approach and findings established in mind-wandering studies can therefore inform this new direction in rsfMRI. For instance, using various introspection techniques, mind-wandering studies have uncovered recurring dimensions of thought content (Sormaz et al., 2018; Turnbull et al., 2019), differentiated between thoughts engaged with and without intention (Seli et al., 2016), identified repeated patterns in the progression of thoughts (Killingsworth and Gilbert, 2010) and emotions (Cowen and Keltner, 2017), and revealed individual differences in dominant modality of thoughts (Delamillieure et al., 2010). These separate lines of mind-wandering research provide unique insights into the characterization of our ongoing experience, but these experiences tend to be measured during an externally oriented task. The specific constraints imposed by each task likely alter the nature of ongoing experience in a manner similar to observer effects, potentially limiting the generalizability of their experiential and neural findings. Therefore, applying these methods in the task-free context of rsfMRI may provide more representative characterizations of the mental and neural patterns underlying naturalistic ongoing experience.
Although the controversy remains surrounding the definition of mind wandering in terms of thought dynamics (Christoff et al., 2016) versus relation to an ongoing task (Seli et al., 2018), this field shares complementary aims with rsfMRI studies to reveal the nature of our ongoing thoughts. Therefore, these two fields of mind wandering and rsfMRI are well positioned to share methods to gain a deeper understanding of how our minds wander during task performance, and how our ongoing experience unfolds in task-free environments, respectively.
While recent neuroimaging studies of mind wandering have adopted a multidimensional version of thought sampling, namely, multidimensional experience sampling (MDES) (Medea et al., 2018; Wang et al., 2018a), earlier work that strictly conceptualized mind wandering as attention away from the task at hand usually included one question per thought probe inquiring about whether or not attention was focused on the task (Christoff et al., 2009). MDES extends beyond this standard question to provide insights into the modality of ongoing experience (e.g., visual, auditory), temporal focus (e.g., past, present, future), and emotional valence (e.g., positive, negative). Responses to MDES probes can be entered into machine learning algorithms to discover groupings (e.g., instances of ongoing experience with similar characteristics) and latent axes (e.g., dimensions that vary in synchrony) (Sormaz et al., 2018; Turnbull et al., 2019, 2020). These latent dimensions can describe meaningful, coordinated aspects of ongoing experience that better explain neuroimaging findings. For example, using principal component analysis (PCA) on 13-dimensional MDES data, Sormaz et al. (2018) identified level-of-detail, task-relatedness, modality, and emotional content as important aspects of ongoing thoughts during active task states. Importantly, the level-of-detail dimension was significantly associated with activity in the default mode network, suggesting that this network's contribution to cognition is not restricted to task-unrelated processes (e.g., the resting state). What this study and others (Smallwood et al., 2016; Wang et al., 2020a) highlight is that, despite MDES' reliance on targeted survey items, it is still possible to combine MDES with advanced mathematical algorithms to uncover combinations of surveyed characteristics about ongoing experience that correlate with neuroimaging data. Nonetheless, it is important to note that such derived dimensions (e.g., the principal components of a PCA) are still initially limited to linear combinations of surveyed items. Therefore, the use of survey items in MDES is still susceptible to missing additional meaningful dimensions completely orthogonal to those initially hypothesized and therefore, included in the MDES probes. Although researchers could also rely on nonlinear methods (e.g., T-distributed stochastic neighbor embedding (T-SNE), neuronal networks), resulting representations might be hard to interpret. Further, survey items compared with open-ended questions may also overlook previously unknown dimensions. Perhaps a more beneficial approach, especially in initial exploratory studies, would be to combine MDES with less constrained forms of “in-the-moment” introspection (e.g., open-ended questions) to facilitate the discovery of novel, unrelated, meaningful dimensions of resting mental life.
One such candidate is an introspection technique known as descriptive experience sampling (DES), developed by Russell Hurlburt (Hurlburt and Akhter, 2006; Hurlburt and Heavy, 2006). The objective of DES is to attain high-fidelity, untainted descriptions of momentary instances of inner experience. In DES terms, inner experience denotes “ongoing naturally occurring thoughts, feelings, sensations, and so on that appear directly before the footlights of consciousness” (Hurlburt et al., 2017), a definition that broadly matches our conceptualization of ongoing experience. According to the DES protocol, subjects are presented with random beeps throughout the day, similar to the aforementioned thought probes. Upon hearing a beep, subjects are instructed to attend to their inner experience at the moment of the beep, and jot down a few notes about it. In contrast to MDES, subjects are free to annotate any aspect of their experience they consider relevant, and thereby are not restricted to characterize their experience based on a set of a predefined questions. Within 24 h from the time those notes were taken, the subject and experimenter would then meet to discuss the notes to uncover a more detailed and undistorted description of each annotated experience (Hurlburt and Akhter, 2006; Hurlburt and Heavy, 2006). During these meetings, subjects are iteratively trained on crucial “experience apprehending” skills to report undisturbed instances of the “raw” experience as it was experienced, untainted by interpretation or inferences. These skills include focusing on the experience right before the beep, reporting only what was actually experienced (as opposed to their interpretations of it), and avoiding coloring descriptions with presumptions, metaphors, or generalities. Once a sufficient number of high-fidelity descriptions have been accumulated, the experimenter extracts salient characteristics of inner experience for each individual, which are those that consistently permeate a given subject's inner experiences. These individual reports are then aggregated for group comparisons (Hurlburt and Heavy, 2006). Importantly, DES has revealed qualitative aspects of ongoing experience that were surprising to both experimenters and participants (e.g., the actual frequency of negative thoughts, the rare incidence of inner speech for some people) (Hurlburt and Heavy, 2006; Kuhn et al., 2014), highlighting how experience sampling can reveal new phenomena to explore.
Although DES was not originally designed for neuroimaging experimentation, this introspection method has been recently adapted and combined with fMRI to evaluate its neuroscientific value (Kuhn et al., 2014; Hurlburt et al., 2016, 2017). Subjects were first trained in DES procedures for 4 d outside of the scanner. Official data collection involved DES during nine 25-min-long rsfMRI scans, followed by meetings after each scan wherein the subject and experimenter discussed subjects' annotated experiences. The authors found that instances of inner speech, and not inner hearing, recruited the left inferior frontal gyrus, a key region involved in the production of overt speech (Kuhn et al., 2014). These results suggest that DES can help make important subtle distinctions about specific characteristics of ongoing experience in relation to their neural correlates.
Collectively, these separate lines of research converge on the scientific value of introspection in revealing details of individuals' ongoing experiences, and demonstrate that, when combined with neuroimaging data, introspection can deliver unique insights about the inner workings of the brain not attainable otherwise (Northoff and Heinzel, 2006; Horikawa et al., 2013; Martinon et al., 2019). Notably, devising an approach that optimally merges these two disciplines and maximizes their capacity for new discoveries is not a simple endeavor, but one determined by the research question at hand. For example, within the framework of mind-wandering research, in addition to the methods outlined above, researchers can also rely on behavioral methods (e.g., sustained attention response tasks) (Rosenberg et al., 2013) to detect and characterize ongoing thought away from the task at hand (for a review on the use of experience sampling methodologies to study mind wandering, see Martinon et al., 2019).
It is worth mentioning that the taxonomy of introspective methods presented here is not a comprehensive list of all scientifically valid introspective methodologies, but one that focuses on methods that could potentially benefit our understanding of resting-state functional connectivity. For example, in addition to retrospective methods and online methods (e.g., what we refer here as in-the-moment methods), Martinon et al. (2019) also discussed the use of dispositional methods (i.e., those that target traits linked to different types of experiences). Similarly, readers should be aware that the two dimensions highlighted here when describing introspective methodology, namely, retrospective versus in-the-moment and open-ended versus targeted survey items, can be manipulated independently (e.g., in-the-moment introspection methods can include open-ended or targeted survey items). As a case in point, while thought probes often tend to use targeted items to survey the ongoing experience, an MDES approach that involves thought probes using open-ended questions (as is the case with DES) can help reveal important dimensions of ongoing experience. For example, Baird et al. (2011) used such an approach to uncover temporal focus and self-relatedness as important characteristics of thoughts during mind wandering. These findings informed subsequent studies about these novel dimensions of thoughts, which have since been directly targeted by specific MDES items in studies of ongoing thoughts and mind wandering (Ruby et al., 2013; Sormaz et al., 2018). Finally, it is important to reiterate that no single introspective method discussed here is perfect for all empirical investigations of ongoing experience. The choice of the optimal introspective method depends heavily on the research question of interest. Here, our objective is to understand the role that ongoing mentation plays in shaping resting-state functional connectivity. This sets some specific demands on how to acquire, process, and merge introspective and neuroimaging data that we discuss next.
Introspection in rsfMRI research
So far, we argued that researchers interested in better interpreting rsfMRI data should explore the potential relationship between coactivation events and simultaneous instances of ongoing experience. In this section, we discuss considerations for implementation of this approach and potential outcomes for such a research agenda. We put greater emphasis on introspection methods relevant for rsfMRI, because a relatively extensive literature (Tagliazucchi et al., 2011; Karahanoğlu et al., 2013; Liu and Duyn, 2013; Liu et al., 2013, 2018; Petridou et al., 2013; Allan et al., 2015; Karahanoğlu and Van De Ville, 2015), with accompanying analytical tools (Caballero Gaudes et al., 2013, 2019; Bolton et al., 2020), already exists on the topic of fMRI coactivation events. Nonetheless, improvements in the sensitivity, reliability, and accuracy of coactivation event detection methodology should be considered as equally important for the advancement of the research agenda proposed here. For example, novel hemodynamic deconvolution methods that are able to account for inter-regional hemodynamic response differences (Wu et al., 2013) or to exploit echo-dependence profiles (Caballero-Gaudes et al., 2018, 2019) may greatly improve our ability to detect cognitively relevant coactivation events that may be strongly linked to experiential characteristics captured with introspective thought probes.
How to effectively annotate rsfMRI with meta-data about ongoing experience
In order to effectively gather and use rich introspection data, researchers will need to consider a few implementational factors specific to rsfMRI research. First, researchers must consider what the goal of their investigation is: to establish statistical relationships at the trait level, to uncover direct “one-to-one” links between contemporaneous instances of brain activity and ongoing experience, or both. This is important because no single introspective approach is optimally suited for all three scenarios. Retrospective techniques may be better suited for studies targeting trait-level relationships or in scenarios where observer effects are an important concern. Conversely, in-the-moment methods will be required when trying to establish instance-level links (e.g., do coactivation events with strong limbic and auditory components signals periods of ongoing experience characterized by inner speech and strong emotional valence?) and temporal blurring effects must be avoided. Despite these core differentiations, researchers should be aware that the undesirable effects of each method can be lessened via astute experimental designs. For example, the sparsity of coactivation events implies that a small number of concise, strategically placed introspective samples, which would minimize observer effects and conserve limited scan time, may suffice when seeking to establish a relationship between ongoing cognition and coactivation events. Notably, real time-fMRI could be used to trigger introspective probes only when specific coactivation events occur. This would maximize sampling efficiency relative to random sampling by ensuring all samples temporally align with coactivation events of interest. Importantly, probe triggers could be inhibited if subjects have been disturbed recently to ensure they can get back to rest in between probes. Similarly, administration of retrospective questionnaires inside the scanner following each individual resting scan will reduce the delay between when an ongoing experience occurs and when it is reported, which may help minimize temporal blurring effects. Ultimately, combining retrospective and in-the-moment approaches will be necessary to circumvent each method's limitations and converge on a detailed model of how ongoing experience shapes rsfMRI functional connectivity and behavior across timescales. For example, recent results in mind-wandering research highlight the complementary nature of different thought sampling techniques by showing that momentary regulation of off-task thought and trait-level propensity to engage in off-task thoughts when task demands are low are both associated with activity in the ventral attention network (Turnbull et al., 2019, 2020).
Second, researchers should account for the limitations imposed by the indirect nature of fMRI as a marker of neural activity. On average, there is a 5 s hemodynamic delay between the onset of neural activity and peak in fMRI response. Researchers should take this into account when fusing in-the-moment introspective information and neuroimaging data, and also when writing such probe questions. Additionally, researchers should be mindful that fMRI provides a systems-level overview of brain activity. Consequently, probes should focus on aspects of ongoing experience likely to be captured at this meso-scale level of detail, such as the modality of imagery (e.g., visual vs auditory) or the emotional valence of a thought, instead of questioning finer-scale details (e.g., whether a thought involved a sibling or colleague) that may not be accessible at current in vivo imaging resolutions.
Third, ongoing experience unfolds in a high dimensional space difficult to characterize via short, concise probes. Structured probes limited to previously established dimensions of mental life: for example, modality (Gorgolewski et al., 2014), content (Gorgolewski et al., 2014; Gonzalez-Castillo et al., 2019), time (Vatansever et al., 2020), and spontaneity (Wang et al., 2018a), may paint an incomplete picture and bias subjects to ignore previously overlooked, yet potentially relevant, aspects of their ongoing experiences. Past studies have successfully used this approach to describe robust patterns of ongoing experience that generalize across individuals (Smallwood et al., 2016), between the laboratory and real life (Ho et al., 2020), and correlate with state and trait-like neural patterns (Vatansever et al., 2020). Yet, as mentioned earlier, even advanced mathematical modeling techniques (e.g., PCA, canonical correlation analysis) (Wang et al., 2018b) cannot uncover additional meaningful but unanticipated dimensions that are completely orthogonal to the aspects of ongoing experience included in a given set of structured probes. Contrariwise, unstructured questioning may result in excessively heterogenous descriptions that are hard to quantify and aggregate, although modern text analysis and machine learning methods have great potential for augmenting traditional experimenter-dependent analysis strategies for extracting common factors from self-report data (Cowen and Keltner, 2017). Since no individual method is a panacea, initial studies may benefit from combining both structured (rating-scale) and unstructured questions (open-ended). To ensure brevity, survey items could be randomized across probes. As key dimensions emerge, subsequent studies can craft structured probe questions relevant to these key dimensions. In addition, efforts dedicated to probe standardization could considerably facilitate data aggregation across studies. That said, it is important to note that reports of fMRI combined with open-ended methodologies (e.g., DES) are much less common than reports in which fMRI data are annotated with information about ongoing experience gathered via targeted survey methods (e.g., MDES). This imbalance in the literature may signal how challenging it might be to correctly implement open-ended methods, so that they can reliably inform fMRI data interpretation. Future research should elucidate whether the proposed benefits of using open-ended methods to annotate and interpret resting-state fMRI are indeed attainable, and to what extent they do provide added value beyond that of targeted survey items.
Fourth, uncovering meaningful relationships between neuroimaging and introspective data will require advanced mathematical models: for example, canonical correlation analysis (Wang et al., 2020b) and spectral embeddings (Gonzalez-Castillo et al., 2019) that need quantitative data as inputs. fMRI data are inherently quantitative, yet introspection data are not and therefore need to be coded into numerical format. The rules governing those transformations may not always be obvious. As the coding approach can impact outcomes and interpretations, adoption of unambiguous and agreed-on coding practices for introspection data are an important part of standardization efforts. For example, during the design of quantitative probes, researchers should carefully consider, and explicitly acknowledge, whether a nominal scale (wherein order does not matter) or ordinal scale (wherein order is considered during modeling) is appropriate for a given observed variable/questionnaire item. This would ensure consistency across different studies using a given questionnaire item.
Finally, it is important to recognize that scientifically valid introspection requires substantial subject training to minimize the effort required to apprehend ongoing experience so as to avoid distorting the target experience with the apprehension process. Appropriate training is also essential to ensure subjects report their “raw” ongoing experience in the moment, without preconceptions, interpretations, and generalities (Price and Barrell, 2012; Hurlburt et al., 2016).
Expected outcomes and limitations
One key premise in our argument, initially supported by the evidence already discussed, is that ongoing conscious mentation plays an important role in the resting state. Should our hypothesis be confirmed, this would have broad implications for the field of rsfMRI. First, it would positively settle open debates regarding the value of time-varying functional connectivity to study brain function. Second, it would emphasize the need to account for systematic intergroup differences in ongoing mentation when interpreting rsfMRI results. Unaccounted for intersession and intersubject variability in ongoing experience during rest might explain why phenotypic prediction (e.g., fluid intelligence, clinical symptoms) is higher for structured experimental designs (e.g., task-based) (Greene et al., 2018) than for rsfMRI. It would also suggest that successful rsfMRI biomarker-based development must find ways to determine the clinical relevance of this source of variance, and either model or remove it accordingly. Third, it would provide an explanatory framework for the relationship between resting-state functional connectivity and task-based activity. Although this relation has been repeatedly observed empirically (Smith et al., 2009; Tavor et al., 2016), its mechanism remains elusive. One hypothesis is that task-evoked activity flows through intrinsic functional connectivity networks (Cole et al., 2016). An alternative explanation, based on our current working hypothesis, would be that such a strong relationship exists because subjects engage in many different “task-like states” when left to rest inside the scanner bore. Finally, a clearer understanding of the link between ongoing experience and rsfMRI data will likely come accompanied by increased knowledge about the functional role of spontaneous neuronal activity and could, ultimately, provide new avenues to explore the neuronal correlates of consciousness.
Alternatively, this proposed research using introspection might eventually establish that perception, cognition, emotion, or other aspects of ongoing experience have negligible, if any, contributions to rsfMRI. Such a negative outcome would still decrease interpretational uncertainty in rsfMRI by eliminating one plausible factor. Indeed, obviating all aspects of conscious mentation accessible to introspection as potential sources of rsfMRI variance would constrain rsfMRI interpretations to nonconscious processes, such as autonomic or homeostatic functions (e.g., breathing, heart rate, metabolic regulation), thereby shifting future research priorities toward basic physiology and invasive animal studies.
In conclusion, rsfMRI sits at the core of large neuroscientific research efforts, such as mapping the human functional connectome, understanding the neural correlates of clinical symptoms, or tracking brain development across the lifespan. Indeed, rsfMRI is a key component of large international neuroimaging initiatives, such as the Human Connectome Project (Van Essen et al., 2012) and the UK Biobank (Miller et al., 2016). Accordingly, the stakes are too high to accept the current levels of interpretational uncertainty that plagues the field. Here we argue that researchers can move the field forward by making first-person description of inner experience the center of our investigational efforts. Triangulation of first-person (i.e., inner experience) with third-person (i.e., neuroimaging) data to better understand the human brain has proven invaluable for other areas of neuroscientific investigation (e.g., emotions, perception). Likewise, to understand rsfMRI data, we recommend asking your participants.
Footnotes
J.G.-C. and P.A.B. were supported by the Intramural Research Program of the National Institute of Mental Health (annual report ZIAMH002783). J.W.Y.K. was supported by the Natural Sciences and Engineering Research Council Canada. C.W.H. was supported by National Institute of Neurological Disorders and Stroke R37NS21135.
The authors declare no competing financial interests.
↵1 Systemic oscillations: signal oscillations present in both cerebral and peripheral vasculature. Their concurrent appearance in peripheral vasculature suggests a non-neuronal origin.
↵2 Echo-time dependence: BOLD-like signal fluctuations in fMRI recordings show a linear dependence with echo time, namely, the lag between excitation and readout pulses, when examined in units of signal percent change. This linear dependence does not occur for non-BOLD-like fluctuations, such as those due to scanner instabilities, head motion, or inflow effects.
↵3 Form: refers to the main modality in which a given experience is perceived by the individual. Examples of form include words, images, and speech.
↵4 Spontaneity: characteristic of a given inner experience that describes the degree to which such experience was initiated without intention, in the absence of top-down control, and/or relationship to any external stimuli.
↵5 Discontinuity of mind: one of the seven dimensions of resting-state cognition identified by Diaz et al. (2013) that refers to mental states described as feeling restless, having busy thoughts, or thoughts that rapidly shift from topic to topic.
↵6 Theory of mind: one of the seven dimensions of resting-state cognition identified by Diaz et al. (2013) that refers to mental states that involve attributing beliefs, emotions, and intents to ourselves and others.
↵7 The exact definition of mind wandering remains controversial, with the majority of studies characterizing mind wandering in relation to an ongoing externally oriented task, and not during resting state, which by definition occurs in absence of an external task. For an in-depth review of this issue, see Seli et al. (2018).
- Correspondence should be addressed to Javier Gonzalez-Castillo at javier.gonzalez-castillo{at}nih.gov
