Research Articles, Behavioral/Cognitive

Optimal Control Costs of Brain State Transitions in Linear Stochastic Systems

Shunsuke Kamiya, Genji Kawakita, Shuntaro Sasai, Jun Kitazono and Masafumi Oizumi
Journal of Neuroscience 11 January 2023, 43 (2) 270-281; https://doi.org/10.1523/JNEUROSCI.1053-22.2022

Abstract

The brain is a system that performs numerous functions by controlling its states. Quantifying the cost of this control is essential as it reveals how the brain can be controlled based on the minimization of the control cost, and which brain regions are most important to the optimal control of transitions. Despite its great potential, the current control paradigm in neuroscience uses a deterministic framework and is therefore unable to consider stochasticity, severely limiting its application to neural data. Here, to resolve this limitation, we propose a novel framework for the evaluation of control costs based on a linear stochastic model. Following our previous work, we quantified the optimal control cost as the minimal Kullback-Leibler divergence between the uncontrolled and controlled processes. In the linear model, we established an analytical expression for minimal cost and showed that we can decompose it into the cost for controlling the mean and covariance of brain activity. To evaluate the utility of our novel framework, we examined the significant brain regions in the optimal control of transitions from the resting state to seven cognitive task states in human whole-brain imaging data of either sex. We found that, in realizing the different transitions, the lower visual areas commonly played a significant role in controlling the means, while the posterior cingulate cortex commonly played a significant role in controlling the covariances.

SIGNIFICANCE STATEMENT The brain performs many cognitive functions by controlling its states. Quantifying the cost of this control is essential as it reveals how the brain can be optimally controlled in terms of the cost, and which brain regions are most important to the optimal control of transitions. Here, we built a novel framework to quantify control cost that takes account of stochasticity of neural activity, which is ignored in previous studies. We established the analytical expression of the stochastic control cost, which enables us to compute the cost in high-dimensional neural data. We identified the significant brain regions for the optimal control in cognitive tasks in human whole-brain imaging data.

Introduction

The brain is a highly complex dynamical network that flexibly transitions to various states to execute a myriad of functions (McKenna et al., 1994; Breakspear, 2017; Kringelbach and Deco, 2020). In this regard, the brain can be considered a system that modulates its internal states to desired states, in accordance with the function the individual needs to perform (Botvinick and Cohen, 2014; Gu et al., 2015). Among the many transitions that bring the system into the various states it requires, some state transitions are more difficult to control than others, depending on the dynamical properties of the neuronal systems. In other words, controlling transitions to some states incurs greater “costs” than controlling transitions to others. Providing a theoretical framework for quantifying transition costs, or control costs, is important for evaluating the difficulty of the shifts between various brain states, and possibly in explaining cognitive loads (Braun et al., 2021; Kawakita et al., 2022), sleep-awake differences (Deco et al., 2019), habituation of cognitive tasks (Szymula et al., 2020), and psychiatric disorders (Braun et al., 2021) with a quantifiable measure. Therefore, the development of such a framework for quantifying control cost in the brain is a vital topic in neuroscience.

A rigorous and promising framework to assess control cost was provided by an approach using control theory, which was first introduced in neuroscience in a pioneering work by Gu et al. (2015). Control theory provides theoretical tools for investigating the dynamical properties of complex systems (Liu et al., 2011; Liu and Barabási, 2016), and its application to neuroscience is opening new doors to mechanistically explaining neural behaviors from brain structures (Gu et al., 2017; Kim et al., 2018; Stiso et al., 2019; Szymula et al., 2020; Cornblath et al., 2020; Braun et al., 2021). However, despite being a strong approach, this framework does not take account of an important property of neural activity: it neglects noise or stochasticity in neural systems. Since neural noises are known to be ubiquitous in the brain and to play critical roles in information processing (Rieke, 1999; Faisal et al., 2008), overlooking the stochasticity of neural systems may result in an inaccurate estimation of the control costs.

In this study, we propose a novel framework to quantify control costs in linear stochastic neural systems. This framework takes advantage of both the linear control theoretic framework (Gu et al., 2015; Kim et al., 2018; Szymula et al., 2020) and the control cost proposed in our previous work (Kawakita et al., 2022). That is, we modeled brain dynamics using linear stochastic differential equations, and defined the control cost as the Kullback-Leibler (KL) divergence between the uncontrolled and controlled processes (Fig. 1) as we did in Kawakita et al. (2022). Thanks to the linearity, we can obtain an analytical expression of the control cost in the stochastic system. Furthermore, as we include a control input term in our model, we can identify those brain regions playing a significant role in the control of state transitions.

Figure 1.
Figure 1.

Schematic of our framework for quantification of control costs in the brain in linear stochastic systems. We model a brain state to follow a certain probability distribution π0 at time t = 0 (left, blue ellipse). In uncontrolled dynamics, the brain state stays in the same distribution (right, blue trajectory and blue ellipse). However, in a state transition, the brain dynamics changes so that it reaches a target distribution πT at time t = T (gold ellipse). We call this altered trajectory the controlled process (gold trajectory). To evaluate how close the controlled process is to the uncontrolled one, we use the KL divergence between the two processes as the cost function, marginalized with the initial and the target distributions (blue square). The distributions of the processes are defined on a path space, the space composed of Rn valued continuous functions defined on [0, T]. This type of KL optimization problem on a path space is referred to as the Schrödinger's bridge problem.

In addition to determining the analytical expression, we also showed that we can decompose this expression into the cost for controlling the mean (referred to as mean control cost) and that for controlling the covariance (covariance control cost). We proved that the mean control cost corresponds to the control cost in the previous deterministic setting. The covariance control cost, on the other hand, is the cost of controlling the covariation among the system, which has not been quantified in previous studies in neuroscience.

After formulating the theoretical framework, we then posed the following two questions. First, how important is it to take account of covariance in estimating control cost? Second, what brain areas are significant in controlling brain state transitions? To address these questions, we applied our new method to real neural data. We used whole-brain fMRI BOLD data of 352 healthy adults of either sex, recorded as part of the Human Connectome Project (HCP) (Van Essen et al., 2013). As for the first question, we found that the influence of the covariance control cost was indeed not negligible. And as for the second, we discovered that the lower visual areas and the posterior cingulate cortex (PCC) play important roles in controlling state transitions, but in different ways: the PCC acts in controlling the covariance and the lower visual areas act in controlling the mean in addition to the covariance.

Materials and methods

Theoretical background

Control cost in deterministic systems

Before formulating the control costs in stochastic systems, we start with a conventional deterministic framework and a method for quantifying control cost under this deterministic setting. The dynamics of the brain is modeled with an n-dimensional state space model, where a brain state x ∈ Rn at time t ∈ [0,T] (T > 0) consists of n scalar values that represent the magnitudes of brain activity. Then for each t ∈ [0,T], x(t) is assumed as a point in an n-dimensional Euclidean space. A state transition of the brain is described as a trajectory on which a point travels from one state to another. An uncontrolled transition is described using linear dynamics, such as the following: x˙(t)=Ax(t).(1)

Here, x: [0,T] → ℝn is a vector that represents the magnitudes of activity of all nodes. A ∈ ℝn × n is a connectivity matrix whose elements represent connectivity weights for each pair of nodes.

We consider situations where a brain state switches from an initial state x(0)=x0∈ℝn to a target state x(T)=xT∈ℝn that is different from the state when following its uncontrolled dynamics (Eq. 1). To realize such a transition, we assume that a control input u(t) is given. We can incorporate the control input into the dynamics as follows: x˙(t)=Ax(t) + Bu(t)(2) where u:[0,T]→ℝp is a control input and B∈ℝn×m is an input matrix that determines the nodes assigned with control inputs (n,m∈ℕ). We assume that this system is controllable (i.e., ∀x0,xT there exists u(t) that enables the transition from x0 to xT). Here, we limit ourselves to cases where B = In (the n × n identity matrix), which implies that the input is given independently to all nodes. In other words, we consider the system with an input v:[0,T]→ℝn, x˙(t)=Ax(t) + v(t).(3)

The control cost (also called control energy) Jcont is defined as the total amount of control input required to steer the system from x0 to the target xT and is expressed as follows: Jcont=∫0T||v||22dt,(4) under x(0)=x0, x(T)=xT.(5)

The minimum of this integral represents the input minimally needed to realize the state transition that satisfies the marginal conditions (Eq. 5). It is written as follows: Jcont*=minv∫0T||v||22dt,(6) and referred to as the minimal control cost. The problem of minimizing the control cost is called the optimal control problem in control theory. Hereafter, we call the optimal control cost simply the “control cost” because we are only concerned with the minimal value in this study. We also refer to this cost as the deterministic cost, since this cost is defined on the deterministic model (Eq. 3). This minimum is known to exist, and the optimal value Jcont* is readily solved (see Kamiya et al., 2022). This metric has been used as the control cost in previous studies in neuroscience (Gu et al., 2017; Kim et al., 2018; Stiso et al., 2019; Cornblath et al., 2020; Deng and Gu, 2020; Braun et al., 2021).

Formulation of a stochastic linear model

To take account of noise and fluctuation in brain activity (Rieke, 1999), we make an extension for three characteristics in the control model of the brain: dynamics, transitions, and the control cost (Fig. 2). In this section, we explain these characteristics in detail.

Figure 2.
Figure 2.

Comparison of deterministic and stochastic state transitions in a brain from the perspective of control theory. The Previous framework column describes the deterministic model of state transitions. The uncontrolled process (blue arrow) is expressed as a linear differential equation x˙(t)=Ax(t) (A is a connectivity matrix of the brain network). A brain state transition is modeled as a transition from a point to another in an n-dimensional state space. Along with a control input v, the brain shifts its state to xT, another point in the state space, at time T (dynamics drawn in yellow). The cost function is often set as the time integral of the squared input. The Proposed framework column explains the stochastic model of brain state transitions. Here, the uncontrolled process (blue arrow) and a controlled process (gold arrow) are given as stochastic processes. A brain state transition is viewed as a shift from an n-dimensional probability distribution to another. As a control cost, the KL divergence between the two processes is examined.

First, we model the brain dynamics through a stochastic process instead of conventional deterministic processes as described in Equations 1 and 3. We then model the dynamics without control, or the uncontrolled process described as an ℝn valued Ornstein Uhlenbeck process x(t) (t ∈ [0,T]), dx(t)=Ax(t)dt + Cdw(t),(7) where each element of x(t) corresponds to the magnitudes of activities in a brain region. Here, w(t) is a standard n-dimensional normal Brownian motion, and A∈ℝn×n,C∈ℝn×n (n∈ℕ). We assume CtC to be nonsingular (tC denotes the matrix transpose of C), which is equivalent to rank(C) = n.

Second, to model the control of transitions, we consider these as from a probability distribution to another probability distribution, instead of as the point-to-point transitions that have been considered in previous studies (Gu et al., 2015; Szymula et al., 2020). In the present study, we consider the control of the system between Gaussian distributions from time t = 0 to time t = T. We assume that the distribution of x at t = 0 follows a Gaussian distribution with mean μ0∈ℝn and covariance Σ0 ∈ PSD (n) (set of n-by-n positive semi-definite matrices), denoted by x(0) ∼ N(μ0,Σ0) (we call this the initial distribution). When the system follows Equation 7, the distribution of x(T) is uniquely determined as follows: x(T) ∼ N(eATx(0),eATΣ0etAT + ∫0TeA(T−τ)CtCetA(T−τ)dτ)=:N(mT,ST).(8)

Specifically, if μ0=0, Σ0=∫0∞eAτCtCetAτdτ,(9) the process (Eq. 7) stays in the same probability distribution in ∀t ≥ 0 (steady-state distribution); thus, μ0=mT=0 and Σ0=ST. The existence of the steady-state distribution is guaranteed if the real parts of all the eigenvalues of A are negative. To describe the control of state transitions, we consider altering the dynamics and the probability distribution at time t = T by giving a certain input. We consider steering the system from the initial distribution N(μ0,Σ0) to a given distribution N(μT,ΣT)(μT∈ℝn,ΣT∈PSD(n)) at t = T, which is different from the final distribution N(mT,ST) in the uncontrolled process. We call N(μT,ΣT) a target distribution, and a process that reaches N(μT,ΣT) at t = T a controlled process.

Third, for the control cost, we adopt the KL divergence between the uncontrolled and controlled processes (Dai Pra, 1991; Léonard and Modal, 2014; Chen et al., 2016a; Kawakita et al., 2022). A KL divergence is a metric that measures the closeness between two probability distributions. Thus, the control cost, defined by the KL divergence, measures the closeness between the uncontrolled and controlled processes. This KL divergence is not the one between the two probability distributions of brain states at a particular time point, such as the KL divergence between the initial distribution N(μ0,Σ0) and the target distribution N(μT,ΣT), but rather the divergence between the probability distributions of the entire paths in the uncontrolled and controlled process from time 0 to T.

In this study, we consider the optimal control problem based on the KL divergence. There are many possible controlled processes that take the system to a given target distribution. Among them, we consider the optimal controlled process, whose control cost is minimized. This process is the closest controlled process to the uncontrolled process in the KL sense. To express this mathematically, let us denote the probability distribution induced by the uncontrolled process by P and that by the controlled process (P̃), which are defined on an abstract space composed of a set of ℝn valued continuous functions on [0, T] (denoted by S=C([0,T],ℝn)). The KL divergence between these paths is written as follows: DKL(P̃,P)=∫SlogdP̃dPdP̃.(10) where dP̃dP is a Radon–Nikodym derivative (we consider when P and P̃ are absolutely continuous to each other). Then the probability distribution of the optimal controlled process we seek is the following Q: Q=arg minP̃DKL(P̃,P).(11) with the boundary conditions x(0)∼N(μ0,Σ0), x(T)∼N(μT,ΣT),(12) if the minimum exists. We call the minimum the stochastic control cost, minP̃DKL(P̃,P)=DKL(Q,P)=∫SlogdQdPdQ.(13)

Again, the proposed framework is compared with the conventional deterministic framework in Figure 2.

The optimization problem of minimizing the KL divergence between two stochastic processes (Eq. 11) has been referred to as Schrödinger's bridge problem (Léonard and Modal, 2014; Chen et al., 2016a). This problem was originally proposed by Erwin Schrödinger for finding the most probable path that moving particles take (Schrodinger, 1931). The Schrödinger bridge problem was subsequently found to be equivalent to an optimal control problem, and has been studied in the field of control theory (Dai Pra, 1991; Léonard and Modal, 2014; Chen et al., 2016b). To our knowledge, our recent study (Kawakita et al., 2022) is the first work in neuroscience to use the KL minimization problem in examining control cost in brain dynamics.

Equivalence between KL cost and quadratic cost

Next, we observe that KL minimization boils down to an optimal control problem where the cost function is the expectation of the quadratic input with respect to the controlled process (Dai Pra, 1991; Chen et al., 2016a).

It is known from previous studies (Dai Pra, 1991; Chen et al., 2016b) that, to consider the minimal KL divergence DKL(P̃,P) under the boundary conditions (Eq. 12), one has only to search for dynamics that can be described as the next form: dx(t)=Ax(t)dt + v(x(t),t)dt + Cdw(t),(14) where v:ℝn×[0,T]→ℝn is a control input. Moreover, the KL divergence and the next quadratic cost become equal: 2·DKL(P̃,P)=EP̃[∫0T||v||(CtC)−12dt].(15) where EP̃[·] represents the expectation on a probability law P̃ and ||v||(CtC)−12= tv(CtC)−1v. Therefore, to obtain the minimal KL divergence, we need to search for the input v that minimizes the expectation of the quadratic form given in the right-hand side of Equation 15. In other words, the KL minimization problem boils down to an optimal control problem whose cost function is the right-hand side of Equation 15. As defined in the previous section, we define the minimum of Equation 15 as the stochastic control cost.

The discussion in this section is stated in more rigorous form in Kamiya et al. (2022).

Experimental design and statistical analysis

For our data processing techniques, see Figure 3 for schematic explanation.

Figure 3.
Figure 3.

Data acquisition techniques. a, The fMRI BOLD data in HCP underwent minimal preprocessing and were then divided into 100 ROIs. b, This gives us 100-dimensional time-series data in our hands. c, The data at rest and under each task state were approximated as a Gaussian distribution in the 100-dimensional space whose individual coordinates represent the magnitude of activity of an ROI. d, To estimate the matrices A and C that determine the resting state dynamics, we first employed a regression using the sparse VAR(1) process (left panel). The coefficient b̂ and E[εt tε] were regressed so that the dynamics has μrest≒0and∑rest as the mean and covariance. This VAR(1) process was then transformed into the corresponding continuous time dynamics using Eq. (18) and Eq. (19) (right panel).

On fMRI data

The 3T fMRI data of 990 subjects were obtained from the Washington University-Minnesota Consortium HCP (Van Essen et al., 2013). To remove the effects of outliers, we picked data of 352 subjects of either sex according to criteria suggested by Ito et al. (2020), (Fig. 3a).

Data preprocessing

Minimally preprocessed fMRI data were used for the resting state and seven cognitive task states (emotion, gambling, language, motor, relational, social, and working memory). Denoising was performed by estimating nuisance regressors and subtracting them from the signal at every vertex (Satterthwaite et al., 2013). For this, 36 nuisance regressors and spike regressors were used following a previous study (Satterthwaite et al., 2013), consisting of (1-6) six motion parameters, (7) a white matter time series, (8) a CSF time series, (9) a global signal time series, (10-18) temporal derivatives of (1-9), and (19-36) quadratic terms for (1-18). The spike regressors were computed with 1.5 mm movement as a spike identification threshold. After regressing these nuisance time courses, a bandpass filter (0.01-0.69 Hz) was applied, whose upper filter bound corresponds to the Nyquist frequency of the time series. A parcellation process proposed by Schaefer et al. (2018) was used to divide the cortex into 100 brain regions, which reduced the complexity of the following analysis (Fig. 3b).

Estimation of parameters characterizing brain states

We next estimated parameters that characterize resting and task brain states. To apply the framework explained in Formulation of a stochastic linear model, we estimated the next parameters: the mean and the covariance matrix of the initial distribution (μ0 and Σ0), those of the target distribution (μT and ΣT), and the matrices in the uncontrolled dynamics (the drift matrix A∈ℝn×n and the product of the diffusion matrix SC:=CtC). As for the diffusion matrix, we estimated SC=CtC∈ℝn×n rather than C itself, as SC is sufficient for cost computation. The detailed methods to estimate the parameters of the initial and the target distributions are covered in Distribution of the resting state and task states, and those to estimate A and SC in Estimation of parameters of the resting state dynamics. For statistical robustness, a bootstrapping method was adopted, where the estimation described below was done using data of 100 randomly chosen subjects of 352 overall subjects, and repeated 100 times independently. The data of the 100 subjects were concatenated to obtain a single time series that is long enough for statistically reliable estimation.

Before estimating the parameters using the concatenated data across different subjects, we need to normalize the time-series data. This normalization is necessary for the following reason. Basically, the absolute values of fMRI BOLD signals are meaningless by themselves because of the bias caused by the water content or the basic blood flow in the brain tissues (Poldrack et al., 2011). Because of this, one cannot directly compare values of BOLD signals in different subjects. Thus, we cannot concatenate data of different subjects using the preprocessed fMRI BOLD signals as they are. To concatenate BOLD signals across subjects, we need some kind of scaling, or normalization, of the data.

We performed the scaling based on the assumption that the sum of the time-series variances in the whole ROIs are constant when a subject is not engaged in a cognitive task. First, for each subject, we computed the trace values of the empirical covariance matrices of the resting state and the task-free moments in each task. Then, we divided the whole time-series data of the rest and each task by the square root of the trace value of the rest and by that of each task's task-free moment, respectively. This division makes the magnitude of the sample covariance matrix (i.e., the trace of the sample covariance matrix) of the rest and the task-free moments to be one in all subjects and tasks. After this normalization, we concatenated the time-series data of the whole resting state and the task-free moments of each task state of all 100 subjects. Using the concatenated time-series data, we estimated the parameters as follows.

Distribution of the resting state and task states

To estimate the resting state distribution, the empirical mean and covariance of the whole time series were used to represent the probability distribution. The empirical mean of the resting state was almost a 100-dimensional zero vector in each subject because of a zero-mean adjustment in preprocessing. To extract the mean of the activity during a task, we did not use the empirical mean and covariance as we did in estimating the resting state distribution. This is because, unlike the resting state data, task time-series data are composed of task-performing and task-free moments. And one needs to know to what extent the activity in the task-performing moments is different from that in the task-free moments. We assumed that, in the task-free moments, the average magnitude of activity is the same as the average of the resting state (i.e., ∼0). Accordingly, to obtain the time-series mean of the activity during the task state, the time-series mean during the task-free moments was subtracted from that in the task-performing moments. The covariance matrix was calculated as the empirical covariance matrix during the task-performing moments. For schematic explanation, see Figure 3c.

Estimation of parameters of the resting state dynamics

In this section, we explain how we estimated the matrices A and SC=CtC that determine the uncontrolled dynamics (Eq. 7). As we mentioned in Results, on application to fMRI data, we regarded the uncontrolled dynamics as the resting state dynamics. Thus, we need to fit the resting state time series to the dynamics equation as follows: dx=Ax(t)dt + Cdw.(16)

When sampled at some interval of Δt > 0, this process is equivalent to the next VAR(1) process as follows: x(t+Δt)=ax(t) + εt+Δt,(17) where a = eAΔt (matrix exponential) and εt+Δt is a zero-mean Gaussian random variable that represents noise. We first estimated the drift term (denoted by â) using the least absolute shrinkage and selection operator regression. A transformation Â=log â/Δt(18) followed to obtain the estimated drift coefficient A in the dynamics.

As for SC=CtC, one can directly infer this from the covariance of εt+Δt (E[εt+Δt tεt+Δt]=:Sεt+Δt) in Equation 17 using the following relationship: SC=−1Δtvec−1[(I−a⊗a)−1log(a⊗a)vec(Sεt+Δt)].(19) where vec is the vectorization operator and vec–1 its inverse, ⊗ the Kronecker product, and log the matrix logarithm. To obtain the estimation of SC (denoted by ŜC), we substituted above the estimated â and the empirical time series covariance Ŝεt+Δt. For schematic explanation, see Figure 3d.

Calculation of entropy of an input map

The entropy S of an input map I= t(I1,⋯,I100)∈ℝ+100 quantifies how dispersed a vector is and takes the maximum if each element Il(l=1,⋯,100) takes the same value. S is calculated by the following: S=−∑l=1100IlIsumlog(IlIsum),(20) where Isum=∑l=1100Il. Note that each element Il(l=1,⋯,100) is positive.

Code accessibility

The codes for computing the stochastic control cost and reproducing the figures on this paper are available at https://github.com/oizumi-lab/SB_toolbox.

Results

Theoretical results

In this section, we show theoretical results obtained from our framework proposed in the previous section. This framework based on the KL minimization was first introduced in neuroscience in our previous study (Kawakita et al., 2022). While the previous study considered a finite-state space, discrete-time stochastic process, we considered a linear continuous-time system in this study. As we have explained in the previous section, there have been many theoretical studies of the linear stochastic system. In addition to what has previously been acknowledged, we newly obtained the following analytical results:

  1. We derived the analytical solution of the control cost. We found that the analytical solution of the stochastic cost can be disintegrated into two portions: the cost of driving the mean (mean control cost) and that for the covariance (covariance control cost).

  2. The mean control cost turns out to correspond to the conventional deterministic control cost in specific occasions. This clarifies the correspondence between the deterministic and the stochastic costs. The covariance control cost has not been quantified in previous applications in neuroscience.

  3. By investigating the control input assigned to each node, we can compute the amount of total input given at one node. This node-level input can also be decomposed into two parts: the input necessary for controlling the mean and that necessary for controlling the covariance.

Analytical solution of the stochastic control cost

We start by showing the analytical solution of the optimal value of Equation 15, which is described as follows: J*= tμdG(T)−1μd− tμTΠ(T)μT+ tμ0Π(0)μ0+∫0Ttr(CtCΠ(t))dt−tr(Π(T)ΣT−Π(0)Σ0).(21)

In the equation, G:[0,T]→ℝn×n,Π:[0,T]→ℝn×n,Ψ:[0,T]×[0,T]→ℝn×n, and μd:=μT−Ψ(T,0)μ0. For the definitions and detailed derivations, see Kamiya et al. (2022). Π(t), G(t), and Ψ(t,s) depend on A, C, Σ0, and ΣT. This gives the analytical expression of the stochastic control cost.

Decomposition of the stochastic cost

We next demonstrate that Equation 21 can be decomposed into two parts: the cost needed to steer the mean (called mean control cost) and the cost needed to steer the covariance (called covariance control cost). More specifically, when we decompose this as follows: J*=Jμ* + JΣ*,(22) where Jμ*= tμdG(T)−1μd− tμTΠ(T)μT+ tμ0Π(0)μ0,(23) JΣ*=∫0Ttr(CtCΠ(t))dt−tr(Π(T)ΣT−Π(0)Σ0),(24) we show that we can consider Jμ* as the cost needed to steer the mean (mean control cost), and JΣ* as the cost needed to steer the covariance (covariance control cost). This decomposition is shown schematically in Figure 4.

We found that the mean control cost Jμ* depends only on the marginal means (μ0 and μT), while the covariance control cost JΣ* depends only on the marginal covariances (Σ0 and ΣT). To explain these dependencies, we start by examining JΣ*. We can see that Equation 24 depends only on the marginal covariances and is independent of the marginal means, since Π(t) relies only on Σ0 and ΣT (in addition to A and C). We can then interpret these terms to represent the cost of navigating the covariance from Σ0 to ΣT. On the other hand, Jμ* (Eq. 23) seems to depend on both the marginal means and marginal covariances, since G and Π are dependent on Σ0 and ΣT. Surprisingly, however, we found that Equation 23 actually does not depend on the marginal covariances, but rather only on the means. Indeed, Jμ* is further rewritten as follows: Jμ*= t(μT−Φ(T,0)μ0)M(T)−1(μT−Φ(T,0)μ0),(25) where Φ is a state transition matrix defined as Φ(t,s)=eA(t−s), and M(T) is a Gramian matrix in the linear stochastic system, M(T)=∫0TΦ(T,τ)CtCtΦ(T,τ)dτ.(26)

For proof of this transformation (Eq. 25), see Kamiya et al. (2022). Thus, Equation 23 can be seen as the cost for controlling the mean from μ0 to μT.

Furthermore, the formulation given in Equation 25 indicates a correspondence between the stochastic control cost and the deterministic control cost (Kamiya et al., 2022). We can easily see that, if the input matrix B is identical to the diffusion matrix C, the mean control cost (Eq. 25) is equal to the deterministic control cost. This correspondence between the mean control cost and the deterministic control cost indicates that the stochastic control cost, defined as the KL divergence, can be considered an extension of the conventional deterministic control cost.

Another merit of the expression of the stochastic cost (Eq. 25) using the Gramian matrix (Eq. 25) is that one can obtain the directions in which controlling the mean is easy or difficult. In the deterministic control system, it is classically known that the eigenvectors of the control Gramian matrix (see Kamiya et al., 2022) can be interpreted as the direction in which the control is easy/difficult (Yan et al., 2015; Brunton and Kutz, 2022). The same logic can be applied to the mean control cost that shares the similar formulation to the Gramian in the deterministic system: the eigenvectors of Equation 26 specify the directions in which the system is good at in controlling its mean.

The covariance control cost, on the other hand, is the cost of shifting the covariance of the probability distribution. This cost has been ignored in previous neuroscience studies (Gu et al., 2015; Kim et al., 2018; Stiso et al., 2019; Szymula et al., 2020; Braun et al., 2021; Singleton et al., 2022). We will cover the neuroscientific implication of the covariance control cost, which can be rephrased as the cost for controlling functional connectivity, in the Discussion. As the formulation Equation 24 is somewhat complicated, an intuitive interpretation of the covariance control cost is not available at this moment.

The significance of this decomposition is that it enables us to separately quantify the influence of taking into account the covariance of brain activities in the control cost (Fig. 4). As we saw, the mean control cost depends only on the marginal means and is shown to correspond to the deterministic control cost under a certain setting. On the other hand, the covariance control cost depends only on the marginal covariances. Thus, the effect of taking account of the covariance of brain activities is reflected only in the covariance control cost.

Input given at each node

Last, we evaluate the contribution of each node to control transitions by the total amount of inputs given at each node in the brain network.

The amount of inputs provided at each node can be calculated as follows. Again, the optimal dynamics is described as follows: dx=Axdt + v*dt + Cdw.(27)

We let v*= t(v1*,⋯,v100*) where vk*:[0,T]→ℝ(k=1,⋯,100). The expectation of the total input given to the k'th node (denoted by I(k),k=1,⋯,100) is then given as follows: I(k)=E[∫0T||vk*||2dt],(28) which is readily transformed into the following: I(k)=(SC∫0T(Π(Σ(t) + μ(t) tμ(t))Π−2m(t) tμ(t)Π + m(t) tm(t))dtSC)kk.(29)

The subscript (·)ij denotes the (i, j)-th entry of a matrix, and m:[0,T]→ℝn is a function for arranging the transient mean. For the derivation, see Kamiya et al. (2022), and m:[0,T]→ℝ is a certain function defined by Kamiya et al. (2022). This value can be decomposed in a similar way to the stochastic control cost (Eq. 22) as follows: I(k)=Iμ(k) + IΣ(k),(30) where Iμ(k)=(SC∫0T(Π(t)μ(t)−m(t)) t(Π(t)μ(t)−m(t))dtSC)kk,(31) IΣ(k)=(SC∫0TΠ(t)Σ(t)Π(t)dtSC)kk.(32)

As m(t) is the function for shifting the mean of the brain dynamics, the first term Iμ(k) can be interpreted as the cost needed to control the mean, which we call the mean input at the k'th ROI. The latter IΣ(k) is the cost of controlling the covariance, which we refer to the covariance input at the k'th ROI.

Results on application to fMRI data

So far, we have formulated a method to quantify the stochastic control cost and contribution of each brain region on state transitions. With this method, we then address the following two problems:

  • How important is it to take account of the covariance of brain state probability distributions?

  • What brain areas are significant in optimally controlling the brain state transitions?

To address the first question (i.e., to assess the influence of covariance), one can compare the covariance control cost JΣ* to the mean control cost Jμ*. We have seen that the mean control cost in the stochastic model corresponds to the optimal control cost in the deterministic model (Decomposition of the stochastic cost), where we consider the control from one point to another point. So imagine, for instance, a case where the mean control cost is far greater than the covariance control cost and hence accounts for the most part of the overall stochastic cost. We might then need to consider only the point-to-point control and might not have to investigate distribution-to-distribution control as we did in the stochastic model. To evaluate the necessity of incorporating probability distributions in our model, we first need to examine to what extent the covariance control cost accounts for the stochastic cost.

To address the second question regarding important brain regions in controlling brain state transitions, we computed control input v for each region as defined in Input given at each node. By decomposing the control input of each region into the mean input (Iμ(k)) and covariance input (IΣ(k)), we can identify regions that are important for shifting the mean and covariance of brain activity. This decomposition enables us to identify not only the brain regions that need to be activated but also those regions that are crucial to the reconfiguration of covariation of whole-brain activity.

To address these questions, we used whole-brain fMRI data recorded as part of the HCP (Van Essen et al., 2013). These data consist of recordings of 990 healthy adults. We used a subset of data composed of 352 subjects based on a criteria proposed by Ito et al. (2020). For each subject, the dataset contains BOLD signals, including a pair of scans at rest and scans under seven different cognitive tasks: emotion, gambling, motor, language, relational, social, and working memory. After a standard preprocessing procedure including denoising with nuisance regressors, the whole voxels were partitioned into 100 ROIs (Fig. 3a,b) following the method proposed by Schaefer et al. (2018).

Figure 4.
Figure 4.

Schematic illustration of the decomposition of stochastic control cost into the mean and covariance control. The stochastic control cost from one state to another is described as a sum of the cost to control the mean and covariance from the initial state to the target state. The radius of the circle represents the magnitude of the mean, while the thickness of an edge represents the covariance value between two nodes.

After preprocessing and parcellation, we applied our framework to the fMRI data. Here, we assumed that the uncontrolled process is the resting state dynamics of the brain and the controlled dynamics is the transition from the probability distribution of the resting state to that of a task state. Under this assumption, we computed the control cost from the resting state probability distribution to a task distribution. To perform this computation, we need to infer two classes of probability distributions: (1) the probability distributions that characterize the resting state dynamics, and (2) the task state probability distribution. For (1), we used a regression using the sparse vector autoregressive (VAR) model (Fig. 3d) on the resting state BOLD data. We assumed the resting state dynamics to be the steady-state dynamics and computed the steady-state probability distribution (N(μ0,Σ0)) and the transition probability distribution (characterized by the matrices A and C). For (2), the mean (μT) and covariance (ΣT) are inferred using the sample mean and covariance in the BOLD signals after normalization (Fig. 3c).

As for the target time constant, we used various T values ranging from 0.1 to 6.0 s, and observed that results did not qualitatively change (Kamiya et al., 2022). Thus, below, we show the results with T = 1.0 s as representative.

To estimate the probability distributions and the stochastic costs, a bootstrapping method was adopted for statistical robustness. The estimation was conducted with 100 randomly chosen subjects of 352 subjects and was repeated 100 times independently (for further details, see Materials and Methods).

Mean and covariance control cost in the stochastic model

Using the estimated probability distributions and the matrices A and C, we computed the stochastic control cost from rest to the seven tasks. For each task, we separately computed the mean control cost (Jμ*), the covariance control cost (JΣ*), and their ratio (JΣ*/Jμ*).

First, to assess the contribution of the covariance control cost compared with the mean control cost, we show the ratio in Table 1. We found that covariance control costs are as large as, or slightly larger than, the mean control costs. The ratios range from 0.4 to 2.8 depending on the task, with an average value of ∼1.5. This result indicates that the contribution of covariance control cost to the overall control cost is as important as that of mean control cost. We also found that these ratios of covariance control cost to mean control cost take various values depending on the specific task. For example, the language task needs a larger covariance control costs than the mean, while the opposite is the case for working memory (WM).

View this table:
Table 1.

Ratios of covariance control compared with the mean control cost in the transition from rest to each taska

To identify the influence of the covariance control cost, we then examined whether incorporating the covariance control cost changes the ordering of the seven tasks by the control costs. We found that incorporating covariance control cost changes the ordering of the seven tasks by the control costs. We show the mean, the covariance, and the stochastic control cost (= mean control cost + covariance control cost) of the seven tasks in ascending order (Fig. 5). The left panel (blue bars) shows the mean control cost values, the middle panel (orange bars) the covariance control cost, and the right panel (yellow bars) the total stochastic control costs. We can see that the order changes when we consider covariance control cost in addition to mean control cost. For example, the mean control cost for the gambling task is larger than that of the motor task; however, when we take account of covariance control cost, this relationship reverses. Thus, we see that the covariance control cost changes the result when estimating the magnitude relationships of control costs.

Figure 5.
Figure 5.

Stochastic control cost from the resting state to task states. Left, Mean control cost values (Jμ*). Middle, Covariance control values (JΣ*). Right, Total control values (J*=Jμ*+JΣ*) in ascending order.

Control inputs at brain regions

To identify which areas are important in optimally controlling brain state transitions, we next computed the amount of inputs given to each brain region (see Input given at each node). We computed the mean input Iμ(k) (Eq. 31) and covariance input IΣ(k) (Eq. 32) at each ROI, obtaining a pair of 100 dimensional vectors ((Iμ(1),⋯,Iμ(100)),(IΣ(1),⋯IΣ(100))) and their sum ((I(1),⋯,I(100))). We can think of these vectors as brain maps of the amount of input required to alter the mean and covariance, and the total inputs. We call these maps the mean input map, covariance input map, and total input map. We computed the inputs and obtained these three maps for each of the seven tasks.

We found that the mean and covariance input maps showed quantitatively different patterns in each task. The mean inputs are typically located in a small number of regions, while the covariance input maps are more widely distributed throughout the brain (Kamiya et al., 2022). We quantitatively evaluated the difference in distributions of the input maps by computing the entropy of the input maps over 100 ROIs for each task. The entropy S of a vector quantifies how dispersed the elements of the vector are; if the inputs are assigned to widely distributed regions, the entropy takes a large value. For a detailed description of how the entropy was calculated, see Materials and Methods. As shown in Figure 6, the entropy of the covariance input maps is larger than that of the mean input maps in all seven tasks. This result suggests that the covariance input maps are more dispersed over the whole brain regions.

Figure 6.
Figure 6.

Entropy values of the mean and covariance input maps for each task. The values were averaged over 100 bootstrap trials.

We then examined the relationships between the two types of control inputs and the changes in BOLD signal magnitudes when transitioning to a task. Intuitively, we may expect that larger inputs are required for brain regions that are activated or deactivated in the task. To study this relationship, we computed the correlation coefficient of the brain activity change and the mean/covariance input maps. We subtracted BOLD signals of preparation periods from those of task periods, and then computed the t values of the differences. We defined the absolute value of the t values as the brain activity change. We found that the mean input maps have high correlation coefficients in all the tasks, whereas the covariance input maps have lower values (Fig. 7). This result indicates that the mean input maps are highly correlated to the changes in BOLD activation level, while the covariance input maps are less related.

Figure 7.
Figure 7.

Correlation coefficients between the absolute values of t values of activation and the input maps. The values were averaged over 100 bootstrap trials.

To identify the brain regions that are commonly important for control of the seven cognitive tasks, we then computed the average of the mean, the covariance, and the total input maps for all seven tasks, as shown in Figure 8a. To calculate the contribution of each task in a fair manner, for each task, we normalized the mean (Iμ(k)), the covariance (IΣ(k)), and the total input map I(k),k=1,⋯,100 by dividing by the sum of the total input to the whole ROIs (∑k=1100I(k)), and then took the average of the seven tasks. To visually inspect particularly important ROIs, we created Figure 8b, which shows ROIs having the largest inputs and explaining 30% of the sum of each input to the whole ROIs (∑k=1100Iμ(k),∑k=1100IΣ(k), or ∑k=1100Iμ(k)). We can see that a large amount of the mean inputs is in the visual areas (Fig. 8a, “mean”). In contrast, the covariance inputs are slightly more distributed over the whole brain. Specifically, a large portion of inputs is concentrated in the visual area, orbitofrontal area, and the PCC (Fig. 8a, “covariance”).

Figure 8.
Figure 8.

Mean, covariance, and total input maps averaged over the seven tasks. a, Average of the mean, covariance, and total inputs for each of 100 ROIs over seven tasks in the HCP dataset. The numerical values indicate percentages compared with the sum of the total input to the whole ROIs. b, The ROIs that account for 30% of the whole mean, covariance, and total inputs shown in a. The numerical values indicate the percentages compared with the sum of the mean, covariance, and total input to the whole ROIs, respectively.

To validate these results on the brain regions that are important in regulating brain state transitions, we counted how many times each ROI appears in the top 10 ROIs with the largest control inputs. Although we identified commonly important regions in controlling transitions by computing the average control inputs of all tasks, the possibility exists that the control input to a certain area is extremely large for one task only and not for the others. To rule out such a possibility and to find those regions that are commonly significant for the seven tasks, we selected the top 10 significant brain regions for each task and counted how many times each region is ranked in the top 10. We performed this analysis for the mean, the covariance, and the total inputs.

This additional analysis further supported our findings that the PCC and lower visual areas are generally important in control transition to task states. The results of the additional analysis are shown in Figure 9. The color of an ROI signifies the number of tasks for which the ROI is ranked in the top 10 (thus; the numbers take integer values from 0 to 7). To facilitate visualization, we only colored ROIs that appeared in the top 10 more than 3 times. For mean input, the lower visual areas are ranked in the top 10 in the majority of the tasks. For the covariance input, the PCC is ranked in the top 10 most frequently, followed by the lower visual areas. For the total input, the lower visual areas and the PCC were ranked most important. Taking these findings together, we conclude that the PCC and the lower visual areas are the most significant regions for transitioning to task states. While the lower visual areas contribute to both the mean and covariance shift, the PCC contributes to shifting covariance only.

Figure 9.
Figure 9.

Number of times an ROI was ranked in the top 10 brain regions that contributed most from rest to the seven tasks. Left to right, each panel shows the areas that contribute to the mean, covariance, and total control, respectively.

Discussion

In this study, we propose a novel framework based on a linear stochastic dynamics to quantify control costs between brain states. We define the minimal KL divergence between the uncontrolled and the optimally controlled paths (Eq. 10) as the stochastic control cost. There are three theoretical contributions in this study. (1) We give a solution for the stochastic control cost (Eq. 21). (2) We show that the cost can be decomposed into two parts: the mean control and covariance control costs (Eq. 22; Fig. 4). (3) The proposed framework allows us to measure the inputs given in each ROI, thanks to the explicit modeling of the input term (Eqs. 28 and 29). The input in each ROI can be further decomposed into inputs necessary for regulating the mean brain activity and the whole-brain functional connectivity (Eq. 30).

We applied our new method to fMRI BOLD data in the HCP dataset. We first found that the influence of incorporating covariance in control costs is not negligible. We further discovered that for regulating the mean, the lower visual areas turned out to be the most significant, whereas for the covariance, the PCC and the lower visual were the most significant.

Stochasticity of control cost

One contribution of the present framework is incorporating stochasticity in quantifying the control cost, which has not been adequately addressed in previous studies in neuroscience (Gu et al., 2015; Kim et al., 2018; Szymula et al., 2020; Braun et al., 2021; Singleton et al., 2022). Our framework incorporates stochasticity in two ways, namely, (1) whether the dynamics of the model is stochastic and (2) whether the cost takes stochasticity into consideration.

Previous studies have not dealt with either of these or considered only the first point. Previous studies that used control theory (Gu et al., 2015, 2017; Kim et al., 2018; Stiso et al., 2019; Cornblath et al., 2020; Szymula et al., 2020; Braun et al., 2021) applied the linear state-space model with deterministic dynamics and deterministic cost (Eq. 6), while another study applied a stochastic model for brain dynamics (Deng and Gu, 2020) but calculated costs based on the deterministic framework.

To our knowledge, our present and recent studies (Kawakita et al., 2022) are the first to take account of stochasticity, not only with regard to brain dynamics but to cost. As we saw in Figure 5, incorporating stochasticity in quantifying the control cost significantly changes the result. Thus, it will be desirable to assess the influence of stochasticity in the brain on control costs as we did in the present work.

Advantages of the decomposition into mean and covariance control cost

One remarkable characteristic of the newly proposed cost is that it allows decomposition into mean and covariance control cost (Eq. 22). A major theoretical advantage of this decomposition is that, as seen in Decomposition of the stochastic cost, it enables us to separately examine the influence when taking covariance into account. Here, we discuss two significant aspects of this decomposition in neuroscience.

First, thanks to the decomposition, the stochastic cost can quantitatively compare the significance of contributions of two separate phenomena in a unified manner, namely, the change in magnitude and covariation of brain activities. Conventionally, changes in the magnitude of brain activities have been assessed through the estimated coefficients (often denoted as β) in the GLM (Friston et al., 1995; Ashby, 2019). The change in covariation has been evaluated through the subtraction of correlation matrices (Cole et al., 2014, 2016) or through network theoretic measures (Braun et al., 2015; Finc et al., 2020; Hahn et al., 2020). The dynamical change in covariation (often referred to as functional connectivity in the field of neuroimaging) is called functional reconfiguration (Kitzbichler et al., 2011; Krienen et al., 2014; Davison et al., 2015; Stitt et al., 2017). In this way, classically, the change in magnitudes and the covariation of activities have been examined separately in different contexts. In contrast, our use of decomposition has allowed us to quantitatively compare control costs for the mean and covariance from a unified perspective.

Second, the proposed framework enables us to investigate regions playing a significant role in the control of brain state transitions, as we have seen in Input given at each node. This topic is covered in the next section.

Significant brain regions in the control of state transitions

The present framework allows us to identify brain areas that contribute to controlling the mean and covariance separately. In Control inputs at brain regions, we saw the general tendencies of the mean and covariance input maps. Compared with the mean inputs, the covariance inputs are (1) more widespread (Fig. 6) and (2) less related to the regions of altered activities (Fig. 7). The first observation might reflect the fact that the alteration of covariance (termed functional reconfiguration) (Kitzbichler et al., 2011; Krienen et al., 2014; Davison et al., 2015; Stitt et al., 2017) occurs all over the brain; thus, broad input to control covariance may be necessary. The second observation means that we cannot estimate important regions for controlling covariance using only the magnitudes of activity. Examining the magnitudes of activity only may lead to the failure to identify significant regions controlling brain state transitions.

On examination of significant areas for control, an intriguing observation is that the PCC commonly contributes to controlling the covariance in many of the tasks examined (Figs. 8a,b, 9, middle panels). Among previous findings related to this result, the PCC is reported to be connected with many brain areas structurally (Hagmann et al., 2008) and functionally (Leech et al., 2012). Although the PCC's specific functions are not yet fully understood, previous studies have revealed that it is associated with many cognitive processes, including cognitive control (Leech and Sharp, 2014), as shown using fMRI (Lin et al., 2016), and through recordings of single-cell firing rates (Hayden et al., 2010). From these studies, it is now considered that the PCC exerts influence on various regions and might thereby play a role in altering the functional connectivity of the brain (Leech et al., 2011; Leech and Sharp, 2014; Lin et al., 2016). Our finding that the PCC is significant in controlling covariance supports this view. Although this speculation needs further evidence, our study showed the importance of the PCC in changing functional connectivity from a control theoretic perspective.

Compared with the PCC and its contribution to controlling covariance, we found that the lower visual areas are included in the significant regions controlling both the mean and covariance (Figs. 8a, 9, left panels). This might be because of the nature of the tasks recorded in the HCP dataset, where subjects were presented visual stimuli as part of the tasks.

Validity and merits of the linear continuous model for fMRI data analysis

In this study, we adopted the linear continuous-state model to describe brain dynamics in the fMRI data. It has been pointed out that a linear model may oversimplify the brain dynamics, which is complex and known to behave in nonlinear manners (Freeman, 1979; Stephan et al., 2008; Rabinovich and Muezzinoglu, 2010). Although we acknowledge this limitation, a recent study has shown that the neural data, including the same fMRI data as we used in this study (HCP), can be better fitted with linear models than nonlinear ones, possibly because of temporal and spatial averaging effects (Nozari et al., 2020). Accordingly, this previous study (Nozari et al., 2020) provides validity for our use of a linear model for the dataset.

To account for the nonlinearity of brain dynamics, another approach to modeling brain dynamics in the fMRI data is to use the discrete-space probabilistic model (Cornblath et al., 2020; Lynn et al., 2021; Kawakita et al., 2022). In our recent study (Kawakita et al., 2022), we used the same KL divergence as control cost in brain state transition as in the present study. The difference from our study lies in that in Kawakita et al. (2022) we used a different probabilistic model, where brain states are discrete.

Although the discrete model can incorporate nonlinearity, the linear model is more feasible than the discrete model in computing control costs in high-dimensional brain dynamics. In the discrete model, the computational cost of the control cost can easily explode, as the control cost is analytically intractable. We have to use iterative algorithms, such as Sinkhorn's algorithms, to compute control costs (Cuturi, 2013; Kawakita et al., 2022); and to lessen the computational burden of the algorithms, we have to coarse-grain and limit the number of brain states. In contrast, control cost in the linear continuous model is analytically tractable, enabling us to compute control costs in high-dimensional dynamics.

Future directions

There are two possible extensions of our work: one is incorporating more general control inputs, and the other is the estimation of actual control costs from neural data.

For the first point, in this study, we considered limited situations of control where we implemented a model in which independent inputs are assigned to all nodes, as discussed in Control cost in deterministic systems. In previous studies that have used the deterministic control theoretical framework (Gu et al., 2015; Kim et al., 2018; Cornblath et al., 2020; Deng and Gu, 2020), the model was grounded on more general cases wherein the system is described with an input term Bu, as in Equation 2. We may be able to consider these general cases of the input matrix B in our stochastic framework with some additional techniques for computation (for details, see Kamiya et al., 2022).

For the second point, we only considered the optimal control cost where brain state transitions are controlled in an optimal manner, with minimization of stochastic control cost. However, in real neural systems, state transitions are not controlled in an optimal manner. An intriguing future direction will be to compare the optimal and actual dynamics using neural data during tasks. Estimating the control cost from real time-series data requires the estimation of time-varying control inputs. This is generally a difficult estimation problem that requires more sophisticated techniques (Milde et al., 2011). Quantifying the actual control cost in real neural data will provide another new insight into cognitive processes, which we plan to pursue in future research.

Footnotes

  • S.K. was supported by Japan Society for the Promotion of Science KAKENHI Grant JP22J23428. M.O. and S.S. were supported by Japan Science and Technology Agency Moonshot R&D Grant JPMJMS2012. M.O. was supported by Japan Science and Technology Agency CREST Grant JPMJCR1864 and Japan Society for the Promotion of Science KAKENHI Grants JP18H02713 and 20H05712.

  • The authors declare no competing financial interests.

  • Correspondence should be addressed to Masafumi Oizumi at c-oizumi{at}g.ecc.u-tokyo.ac.jp

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.

References

  1. ↵
    2. Ashby FG
    (2019) Statistical analysis of fMRI data. Cambridge, MA: Massachusetts Institute of Technology.
  2. ↵
    2. Botvinick MM,
    3. Cohen JD
    (2014) The computational and neural basis of cognitive control: charted territory and new frontiers. Cogn Sci 38:1249–1285. doi:10.1111/cogs.12126 pmid:25079472
    OpenUrlCrossRefPubMed
  3. ↵
    2. Braun U,
    3. Schäfer A,
    4. Walter H,
    5. Erk S,
    6. Romanczuk-Seiferth N,
    7. Haddad L,
    8. Schweiger JI,
    9. Grimm O,
    10. Heinz A,
    11. Tost H,
    12. Meyer-Lindenberg A,
    13. Bassett DS
    (2015) Dynamic reconfiguration of frontal brain networks during executive cognition in humans. Proc Natl Acad Sci USA 112:11678–11683. doi:10.1073/pnas.1422487112 pmid:26324898
    OpenUrlAbstract/FREE Full Text
  4. ↵
    2. Braun U,
    3. Harneit A,
    4. Pergola G,
    5. Menara T,
    6. Schäfer A,
    7. Betzel RF,
    8. Zang Z,
    9. Schweiger JI,
    10. Zhang X,
    11. Schwarz K,
    12. Chen J,
    13. Blasi G,
    14. Bertolino A,
    15. Durstewitz D,
    16. Pasqualetti F,
    17. Schwarz E,
    18. Meyer-Lindenberg A,
    19. Bassett DS,
    20. Tost H
    (2021) Brain network dynamics during working memory are modulated by dopamine and diminished in schizophrenia. Nat Commun 12:3478. doi:10.1038/s41467-021-23694-9 pmid:34108456
    OpenUrlCrossRefPubMed
  5. ↵
    2. Breakspear M
    (2017) Dynamic models of large-scale brain activity. Nat Neurosci 20:340–352. doi:10.1038/nn.4497 pmid:28230845
    OpenUrlCrossRefPubMed
  6. ↵
    2. Brunton SL,
    3. Kutz JN
    (2022) Data-driven science and engineering: machine learning, dynamical systems, and control. Cambridge: Cambridge UP.
  7. ↵
    2. Chen Y,
    3. Georgiou TT,
    4. Pavon M
    (2016a) On the relation between optimal transport and Schrödinger bridges: a stochastic control viewpoint. J Optim Theory Appl 169:671–691. doi:10.1007/s10957-015-0803-z
    OpenUrlCrossRef
  8. ↵
    2. Chen Y,
    3. Georgiou TT,
    4. Pavon M
    (2016b) Optimal steering of a linear stochastic system to a final probability distribution, part I. IEEE Trans Automat Contr 61:1158–1169. doi:10.1109/TAC.2015.2457784
    OpenUrlCrossRef
  9. ↵
    2. Cole MW,
    3. Bassett DS,
    4. Power JD,
    5. Braver TS,
    6. Petersen SE
    (2014) Intrinsic and task-evoked network architectures of the human brain. Neuron 83:238–251. doi:10.1016/j.neuron.2014.05.014 pmid:24991964
    OpenUrlCrossRefPubMed
  10. ↵
    2. Cole MW,
    3. Ito T,
    4. Bassett DS,
    5. Schultz DH
    (2016) Activity flow over resting-state networks shapes cognitive task activations. Nat Neurosci 19:1718–1726. doi:10.1038/nn.4406 pmid:27723746
    OpenUrlCrossRefPubMed
  11. ↵
    2. Cornblath EJ,
    3. Ashourvan A,
    4. Kim JZ,
    5. Betzel RF,
    6. Ciric R,
    7. Adebimpe A,
    8. Baum GL,
    9. He X,
    10. Ruparel K,
    11. Moore TM,
    12. Gur RC,
    13. Gur RE,
    14. Shinohara RT,
    15. Roalf DR,
    16. Satterthwaite TD,
    17. Bassett DS
    (2020) Temporal sequences of brain activity at rest are constrained by white matter structure and modulated by cognitive demands. Commun Biol 3:261. doi:10.1038/s42003-020-0961-x pmid:32444827
    OpenUrlCrossRefPubMed
  12. ↵
    2. Cuturi M
    (2013) Sinkhorn distances: lightspeed computation of optimal transport. Adv Neural Inf Process Syst 26:pp 2292–2300.
    OpenUrl
  13. ↵
    2. Dai Pra P
    (1991) A stochastic control approach to reciprocal diffusion processes. Appl Math Optim 23:313–329. doi:10.1007/BF01442404
    OpenUrlCrossRef
  14. ↵
    2. Davison EN,
    3. Schlesinger KJ,
    4. Bassett DS,
    5. Lynall ME,
    6. Miller MB,
    7. Grafton ST,
    8. Carlson JM
    (2015) Brain network adaptability across task states. PLoS Comput Biol 11:e1004029. doi:10.1371/journal.pcbi.1004029 pmid:25569227
    OpenUrlCrossRefPubMed
  15. ↵
    2. Deco G,
    3. Cruzat J,
    4. Cabral J,
    5. Tagliazucchi E,
    6. Laufs H,
    7. Logothetis NK,
    8. Kringelbach ML
    (2019) Awakening: predicting external stimulation to force transitions between different brain states. Proc Natl Acad Sci USA 116:18088–18097. doi:10.1073/pnas.1905534116 pmid:31427539
    OpenUrlAbstract/FREE Full Text
  16. ↵
    2. Deng S,
    3. Gu S
    (2020) Controllability analysis of functional brain networks. arXiv 2201.02340. https://doi.org/10.48550/arXiv.2201.02340.
  17. ↵
    2. Faisal AA,
    3. Selen LP,
    4. Wolpert DM
    (2008) Noise in the nervous system. Nat Rev Neurosci 9:292–303. doi:10.1038/nrn2258 pmid:18319728
    OpenUrlCrossRefPubMed
  18. ↵
    2. Finc K,
    3. Bonna K,
    4. He X,
    5. Lydon-Staley DM,
    6. Kühn S,
    7. Duch W,
    8. Bassett DS
    (2020) Dynamic reconfiguration of functional brain networks during working memory training. Nat Commun 11:15. doi:10.1038/s41467-020-15631-z
    OpenUrlCrossRef
  19. ↵
    2. Freeman WJ
    (1979) Nonlinear gain mediating cortical stimulus-response relations. Biol Cybern 33:237–247. doi:10.1007/BF00337412 pmid:497266
    OpenUrlCrossRefPubMed
  20. ↵
    2. Friston KJ,
    3. Holmes AP,
    4. Poline J,
    5. Grasby P,
    6. Williams S,
    7. Frackowiak RS,
    8. Turner R
    (1995) Analysis of fMRI time-series revisited. Neuroimage 2:45–53. doi:10.1006/nimg.1995.1007 pmid:9343589
    OpenUrlCrossRefPubMed
  21. ↵
    2. Gu S,
    3. Pasqualetti F,
    4. Cieslak M,
    5. Telesford QK,
    6. Yu AB,
    7. Kahn AE,
    8. Medaglia JD,
    9. Vettel JM,
    10. Miller MB,
    11. Grafton ST,
    12. Bassett DS
    (2015) Controllability of structural brain networks. Nat Commun 6:8414. doi:10.1038/ncomms9414 pmid:26423222
    OpenUrlCrossRefPubMed
  22. ↵
    2. Gu S,
    3. Betzel RF,
    4. Mattar MG,
    5. Cieslak M,
    6. Delio PR,
    7. Grafton ST,
    8. Pasqualetti F,
    9. Bassett DS
    (2017) Optimal trajectories of brain state transitions. Neuroimage 148:305–317. doi:10.1016/j.neuroimage.2017.01.003 pmid:28088484
    OpenUrlCrossRefPubMed
  23. ↵
    2. Hagmann P,
    3. Cammoun L,
    4. Gigandet X,
    5. Meuli R,
    6. Honey CJ,
    7. Wedeen VJ,
    8. Sporns O
    (2008) Mapping the structural core of human cerebral cortex. PLoS Biol 6:e159. doi:10.1371/journal.pbio.0060159 pmid:18597554
    OpenUrlCrossRefPubMed
  24. ↵
    2. Hahn A, et al
    . (2020) Reconfiguration of functional brain networks and metabolic cost converge during task performance. Elife 9:e52443. doi:10.7554/eLife.52443
    OpenUrlCrossRef
  25. ↵
    2. Hayden B,
    3. Smith DV,
    4. Platt M
    (2010) Cognitive control signals in posterior cingulate cortex. Front Hum Neurosci 4:223. doi:10.3389/fnhum.2010.00223 pmid:21160560
    OpenUrlCrossRefPubMed
  26. ↵
    2. Ito T,
    3. Brincat SL,
    4. Siegel M,
    5. Mill RD,
    6. He BJ,
    7. Miller EK,
    8. Rotstein HG,
    9. Cole MW
    (2020) Task-evoked activity quenches neural correlations and variability across cortical areas. PLoS Comput Biol 16:e1007983. doi:10.1371/journal.pcbi.1007983 pmid:32745096
    OpenUrlCrossRefPubMed
  27. ↵
    2. Kamiya S,
    3. Kawakita G,
    4. Sasai S,
    5. Kitazono J,
    6. Oizumi M
    (2022) Optimal control costs of brain state transitions in linear stochastic systems. bioRxiv 490252. https://doi.org/10.1101/2022.05.01.490252.
  28. ↵
    2. Kawakita G,
    3. Kamiya S,
    4. Sasai S,
    5. Kitazono J,
    6. Oizumi M
    (2022) Quantifying brain state transition cost via Schrödinger bridge. Netw Neurosci 6:118–134. doi:10.1162/netn_a_00213 pmid:35356194
    OpenUrlCrossRefPubMed
  29. ↵
    2. Kim JZ,
    3. Soffer JM,
    4. Kahn AE,
    5. Vettel JM,
    6. Pasqualetti F,
    7. Bassett DS
    (2018) Role of graph architecture in controlling dynamical networks with applications to neural systems. Nat Phys 14:91–98. doi:10.1038/nphys4268 pmid:29422941
    OpenUrlCrossRefPubMed
  30. ↵
    2. Kitzbichler MG,
    3. Henson RN,
    4. Smith ML,
    5. Nathan PJ,
    6. Bullmore ET
    (2011) Cognitive effort drives workspace configuration of human brain functional networks. J Neurosci 31:8259–8270. doi:10.1523/JNEUROSCI.0440-11.2011 pmid:21632947
    OpenUrlAbstract/FREE Full Text
  31. ↵
    2. Krienen FM,
    3. Yeo BT,
    4. Buckner RL
    (2014) Reconfigurable task-dependent functional coupling modes cluster around a core functional architecture. Philos Trans R Soc Lond B Biol Sci 369:20130526.
    OpenUrlCrossRefPubMed
  32. ↵
    2. Kringelbach ML,
    3. Deco G
    (2020) Brain states and transitions: insights from computational neuroscience. Cell Rep 32:108128. doi:10.1016/j.celrep.2020.108128 pmid:32905760
    OpenUrlCrossRefPubMed
  33. ↵
    2. Leech R,
    3. Sharp DJ
    (2014) The role of the posterior cingulate cortex in cognition and disease. Brain 137:12–32. doi:10.1093/brain/awt162 pmid:23869106
    OpenUrlCrossRefPubMed
  34. ↵
    2. Leech R,
    3. Kamourieh S,
    4. Beckmann CF,
    5. Sharp DJ
    (2011) Fractionating the default mode network: distinct contributions of the ventral and dorsal posterior cingulate cortex to cognitive control. J Neurosci 31:3217–3224. doi:10.1523/JNEUROSCI.5626-10.2011 pmid:21368033
    OpenUrlAbstract/FREE Full Text
  35. ↵
    2. Leech R,
    3. Braga R,
    4. Sharp DJ
    (2012) Echoes of the brain within the posterior cingulate cortex. J Neurosci 32:215–222. doi:10.1523/JNEUROSCI.3689-11.2012 pmid:22219283
    OpenUrlAbstract/FREE Full Text
  36. ↵
    2. Léonard C,
    3. Modal X
    (2014) A survey of the Schrödinger problem and some of its connections with optimal transport. Discrete Contin Dyn Syst Ser A 34:1533–1574. doi:10.3934/dcds.2014.34.1533
    OpenUrlCrossRef
  37. ↵
    2. Lin P,
    3. Yang Y,
    4. Jovicich J,
    5. De Pisapia N,
    6. Wang X,
    7. Zuo CS,
    8. Levitt JJ
    (2016) Static and dynamic posterior cingulate cortex nodal topology of default mode network predicts attention task performance. Brain Imaging Behav 10:212–225. doi:10.1007/s11682-015-9384-6 pmid:25904156
    OpenUrlCrossRefPubMed
  38. ↵
    2. Liu YY,
    3. Barabási AL
    (2016) Control principles of complex systems. Rev Mod Phys 88:1–61.
    OpenUrlCrossRef
  39. ↵
    2. Liu YY,
    3. Slotine JJ,
    4. Barabási AL
    (2011) Controllability of complex networks. Nature 473:167–173. doi:10.1038/nature10011 pmid:21562557
    OpenUrlCrossRefPubMed
  40. ↵
    2. Lynn CW,
    3. Cornblath EJ,
    4. Papadopoulos L,
    5. Bertolero MA,
    6. Bassett DS
    (2021) Broken detailed balance and entropy production in the human brain. Proc Nat Acad Sci USA 118:e2109889118.
    OpenUrlAbstract/FREE Full Text
  41. ↵
    2. McKenna TM,
    3. McMullen TA,
    4. Shlesinger MF
    (1994) The brain as a dynamic physical system. Neuroscience 60:587–605. doi:10.1016/0306-4522(94)90489-8 pmid:7936189
    OpenUrlCrossRefPubMed
  42. ↵
    2. Milde T,
    3. Schwab K,
    4. Walther M,
    5. Eiselt M,
    6. Schelenz C,
    7. Voss A,
    8. Witte H
    (2011) Time-variant partial directed coherence in analysis of the cardiovascular system. a methodological study. Physiol Meas 32:1787–1805. doi:10.1088/0967-3334/32/11/S06 pmid:22027489
    OpenUrlCrossRefPubMed
  43. ↵
    2. Nozari E,
    3. Stiso J,
    4. Caciagli L,
    5. Cornblath EJ,
    6. He X,
    7. Bertolero MA,
    8. Mahadevan AS,
    9. Pappas GJ,
    10. Bassett DS
    (2020) Is the brain macroscopically linear? A system identification of resting state dynamics. arXiv 2012.12351. https://doi.org/10.48550/arXiv.2012.12351.
  44. ↵
    2. Poldrack RA,
    3. Mumford JA,
    4. Nichols TE
    (2011) Handbook of functional MRI data analysis. Cambridge: Cambridge UP.
  45. ↵
    2. Rabinovich MI,
    3. Muezzinoglu M
    (2010) Nonlinear dynamics of the brain: emotion and cognition. Phys-Usp 53:357–372. doi:10.3367/UFNe.0180.201004b.0371
    OpenUrlCrossRef
  46. ↵
    2. Rieke F
    (1999) Spikes: exploring the neural code. Cambridge, MA: Massachusetts Institute of Technology.
  47. ↵
    2. Satterthwaite TD,
    3. Elliott MA,
    4. Gerraty RT,
    5. Ruparel K,
    6. Loughead J,
    7. Calkins ME,
    8. Eickhoff SB,
    9. Hakonarson H,
    10. Gur RC,
    11. Gur RE,
    12. Wolf DH
    (2013) An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data. Neuroimage 64:240–256. doi:10.1016/j.neuroimage.2012.08.052 pmid:22926292
    OpenUrlCrossRefPubMed
  48. ↵
    2. Schaefer A,
    3. Kong R,
    4. Gordon EM,
    5. Laumann TO,
    6. Zuo XN,
    7. Holmes AJ,
    8. Eickhoff SB,
    9. Yeo BT
    (2018) Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity MRI. Cereb Cortex 28:3095–3114. doi:10.1093/cercor/bhx179 pmid:28981612
    OpenUrlCrossRefPubMed
  49. ↵
    2. Schrodinger E
    (1931) Uber die umkehrung der naturgesetze sitz ber der preuss. Akad Wissen Berlin Phys Math. Klasse, 10, pp 144–153.
    OpenUrl
  50. ↵
    2. Singleton SP,
    3. Luppi AI,
    4. Carhart-Harris RL,
    5. Cruzat J,
    6. Roseman L,
    7. Nutt DJ,
    8. Deco G,
    9. Kringelbach ML,
    10. Stamatakis EA,
    11. Kuceyeski A
    (2022) Receptor-informed network control theory links LSD and psilocybin to a flattening of the brain's control energy landscape. Nat Commun 13:5812. doi:10.1038/s41467-022-33578-1 pmid:36192411
    OpenUrlCrossRefPubMed
  51. ↵
    2. Stephan KE,
    3. Kasper L,
    4. Harrison LM,
    5. Daunizeau J,
    6. den Ouden HE,
    7. Breakspear M,
    8. Friston KJ
    (2008) Nonlinear dynamic causal models for fMRI. Neuroimage 42:649–662. doi:10.1016/j.neuroimage.2008.04.262 pmid:18565765
    OpenUrlCrossRefPubMed
  52. ↵
    2. Stiso J,
    3. Khambhati AN,
    4. Menara T,
    5. Kahn AE,
    6. Stein JM,
    7. Das SR,
    8. Gorniak R,
    9. Tracy J,
    10. Litt B,
    11. Davis KA,
    12. Pasqualetti F,
    13. Lucas TH,
    14. Bassett DS
    (2019) White matter network architecture guides direct electrical stimulation through optimal state transitions. Cell Rep 28:2554–2566.e7. doi:10.1016/j.celrep.2019.08.008 pmid:31484068
    OpenUrlCrossRefPubMed
  53. ↵
    2. Stitt I,
    3. Hollensteiner KJ,
    4. Galindo-Leon E,
    5. Pieper F,
    6. Fiedler E,
    7. Stieglitz T,
    8. Engler G,
    9. Nolte G,
    10. Engel AK
    (2017) Dynamic reconfiguration of cortical functional connectivity across brain states. Sci Rep 7:8797. doi:10.1038/s41598-017-08050-6 pmid:28821753
    OpenUrlCrossRefPubMed
  54. ↵
    2. Szymula KP,
    3. Pasqualetti F,
    4. Graybiel AM,
    5. Desrochers TM,
    6. Bassett DS
    (2020) Habit learning supported by efficiently controlled network dynamics in naive macaque monkeys. arXiv 2006.14565. https://doi.org/10.48550/arXiv.2006.14565.
  55. ↵
    2. Van Essen DC,
    3. Smith SM,
    4. Barch DM,
    5. Behrens TE,
    6. Yacoub E,
    7. Ugurbil K
    , WU-Minn HCP Consortium (2013) The WU-Minn Human Connectome Project: an overview. Neuroimage 80:62–79. doi:10.1016/j.neuroimage.2013.05.041 pmid:23684880
    OpenUrlCrossRefPubMed
  56. ↵
    2. Yan G,
    3. Tsekenis G,
    4. Barzel B,
    5. Slotine JJ,
    6. Liu YY,
    7. Barabási AL
    (2015) Spectrum of controlling and observing complex networks. Nat Phys 11:779–786. doi:10.1038/nphys3422
    OpenUrlCrossRef
Back to top

In this issue

Email
Print
View Full Page PDF
Citation Tools
Respond to this article
Optimal Control Costs of Brain State Transitions in Linear Stochastic Systems
Shunsuke Kamiya, Genji Kawakita, Shuntaro Sasai, Jun Kitazono, Masafumi Oizumi
Journal of Neuroscience 11 January 2023, 43 (2) 270-281; DOI: 10.1523/JNEUROSCI.1053-22.2022
Twitter logo Facebook logo Mendeley logo

Jump to section

Keywords

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.