A Bayesian framework for simultaneously modeling neural and behavioral data☆
Highlights
► We present a method for combining behavioral and neural data into a single analysis. ► The method uses neural data to constrain behavioral models. ► Our method allows for any behavioral model to be combined with any neural model.
Introduction
Currently, there are two main methods to study cognition. The first and oldest method is known as cognitive modeling. Given a set of experimental data, one assumes that observers use a particular process, known as a cognitive model, to produce the observed data. The processes used by the cognitive model are controlled by a set of unknown parameters. The parameters of the cognitive model are then estimated and psychologically meaningful interpretations are based on these parameter estimates. While cognitive models have been effective tools for identifying how cognition changes as a function of task demands, they suffer from being highly abstract representations of what is essentially a system of biological processes. The second method to study cognition is to make contact with the biological substrate more directly and measure brain activity using methods such as positron emission tomography (PET), functional-magnetic resonance imaging (fMRI), electroencephalography (EEG), or diffusion-weighted imaging (DWI), and we will refer to this broad class of data as “neural data”. While neural data provide valuable information about the biological and physical aspects of cognition, traditional neural imaging analyses (e.g., general linear models) are limited because they do not attempt to describe cognitive processes. Because both methods for studying cognition have clear advantages and disadvantages (Wilkinson and Halligan, 2004), there has been a recent surge of interest in combining both sources of information to provide a single explanation of the underlying process (e.g., Anderson et al., 2008, Borst et al., 2011, Dolan, 2008, Forstmann et al., 2008, Forstmann et al., 2010, Forstmann et al., 2011, Gläscher and O'Doherty, 2010, O'Doherty et al., 2007).
In this article, we propose a general framework for describing neural and behavioral data with a single model. Our approach is to treat the two sources of information as separate measurements of the same cognitive construct. To fit the model, we make use of a hierarchical Bayesian approach, which has become an important method for inference in both the neural (e.g., Friston et al., 2002, Gershman et al., 2011, Guo et al., 2008, Wu et al., 2011), and cognitive modeling (e.g., Lee, 2011, Shiffrin et al., 2008) literatures.
Using the hierarchical Bayesian approach provides a number of benefits. First, the Bayesian framework provides meaningful, interpretable information at both the subject and group levels. Second, the Bayesian framework lends itself naturally to principled inclusion of missing data. We will show how our framework allows us to make predictions for missing data, based solely on parameter relationships learned from fitting the model. In particular, we show that we can make informed predictions of behavioral data given only neural data, and vice versa. Third, our framework allows us to infer relationships between parameters, relationships that need not be hypothesized a priori. This feature affords us explorative opportunities in the form of the Bayesian posterior distribution. Fourth, the framework we propose does not require a commitment to any particular model, as in other joint modeling approaches (e.g., Anderson et al., 2008, Borst et al., 2011, Mazurek et al., 2003). By using a hierarchical Bayesian approach, we can choose any particular cognitive model to explain the behavioral data, and any neural model to explain the neural data. Subsequently, our framework links the two models together and simultaneously infers meaningful relationships between the two models while also providing a unifying account of brain and behavioral data.
Using such a framework also provides a method for answering much more general questions, which we do not attempt to answer here. For example, linking brain and behavioral data allows us to directly perform model selection on multiple theories of cognition. One could fit several different cognitive models combined with a single neural model to data, and the joint model that fit the full data set best would be the preferred model. In this way, the neural data provides deeper constraints on cognitive models, and in so doing, can be used to better test cognitive theories.
We first provide a brief introduction to the two different types of measurements, and then describe our joint modeling approach. We then demonstrate the utility of our approach in a simulation study. Finally, we apply our model to data from an experiment containing both neural data and behavioral data that can be fit with a computational model. We show that meaningful relationships between model parameters and neural data can be inferred directly from fitting the model, and these relationships can be further exploited to make predictions about the distribution of missing or unobserved data.
Section snippets
Prior research
Although both behavioral and neural data are central to the study of cognition, few attempts have been made to merge them. Perhaps one of the most successful approaches toward this goal is model-based fMRI analysis (Gläscher and O'Doherty, 2010, O'Doherty et al., 2007). In this procedure, a cognitive model is first used to simulate neural data. To do this, often cognitive models are convolved with particular functions that resemble neural effects, such as the hemodynamic response function that
The framework
We wish to provide a joint explanation for the jth subject's neural Nj and behavioral Bj data. If it is difficult or undesirable to specify the joint distribution of (Nj, Bj) under a single model, we can begin by describing how each individual source should be modeled. We will denote the cognitive model as Behav with unknown parameters θ, and the neural model as Neural with unknown parameters δ. A key benefit of our framework is that we are not limited to a particular cognitive or neural model.
Simulation study
In order to highlight the advantages of our approach we conducted a simulation study in which we generated data from the joint model so that the neural side (i.e., the left side of Fig. 1) consisted of fMRI scans and the behavioral side (i.e., the right side of Fig. 1) consisted of data from a recognition memory task. The simulation was designed to mimic a typical recognition memory experiment in which, during a study phase, a subject is provided with a single set of items (e.g., words or
Fitting the joint model to response time and tract strength data
In this section, we will demonstrate the joint model's ability to generalize and predict future data for new subjects while having only neural data or some combination of neural data and (sparse) behavioral data. We demonstrate this feature of the model on experimental data reported in Forstmann et al. (2011). The study was designed to provide further evidence for the striatal hypothesis of the speed accuracy tradeoff (Bogacz et al., 2010, Forstmann et al., 2008, Forstmann et al., 2010), which
General discussion
We have presented a new approach to unifying neural and cognitive models to better understand cognition. In contrast to previous methods, our method is extremely flexible because it can combine any neural model with any cognitive model. Our method also provides a way to infer the relationships that might exist between the biological and physical properties of the brain and higher-level cognitive processes.
By combining two separate models into a single Bayesian hierarchical model, we can make
Conclusions
In this article, we have presented a hierarchical Bayesian framework for combining both neural and cognitive models into a single unifying model. We have shown that our approach provides a number of benefits over current approaches and allows for principled inference of the relationships between the biological properties of the brain assessed by neuroimaging techniques and the theories of cognition that are used to understand higher levels of cognitive processing. With this approach, one can
References (62)
- et al.
Information-processing modules and their relative modality specificity
Cogn. Psychol.
(2007) - et al.
Using brain imaging to track problem solving in a complex state space
NeuroImage
(2012) - et al.
The neural basis of the speed–accuracy tradeoff
Trends Neurosci.
(2010) - et al.
Using a symbolic process model as input for model-based fMRI analysis: locating the neural correlates of problem state replacement
NeuroImage
(2011) - et al.
The simplest complete model of choice reaction time: linear ballistic accumulation
Cogn. Psychol.
(2008) Neuroimaging of cognition: past, present and future
Neuron
(2008)- et al.
Drawing conclusions from choice response time models: a tutorial
J. Math. Psychol.
(2011) - et al.
Bayesian fMRI data analysis with sparse spatial basis functions
NeuroImage
(2007) Bayesian estimation of dynamical systems: an application to fMRI
NeuroImage
(2002)- et al.
Post hoc Bayesian model selection
NeuroImage
(2011)
Classical and Bayesian inference in neuroimaging
NeuroImage
Dynamic causal modeling
NeuroImage
A tutorial on Bayesian nonparametric models
J. Math. Psychol.
A topographic latent source model for fMRI data
NeuroImage
The mechanism underlying inhibition of saccadic return
Cogn. Psychol.
The basal ganglia: focused selection and inhibition of competing motor programs
Prog. Neurobiol.
Deconvolving bold activation in event-related designs for multivoxel pattern classification analyses
NeuroImage
Modeling individual differences using Dirichlet processes
J. Math. Psychol.
Temporal difference models and reward-related learning in the human brain
Neuron
Bayesian spatiotemporal model of fMRI data
NeuroImage
Approximate Bayesian computation with differential evolution
J. Math. Psychol.
Efficient Bayesian multivariate fMRI analysis using a sparsifying spatio-temporal prior
NeuroImage
An encompassing prior generalization of the Savage–Dickey density ratio
Comput. Stat. Data Anal.
A hierarchical Bayesian approach for learning sparse spatio-temporal decompositions of multichannel EEG
NeuroImage
How Can the Human Mind Occur in the Physical Universe?
Using fMRI to test models of complex cognition
Cogn. Sci.
Neural imaging to track mental states
Proc. Natl. Acad. Sci. U. S. A.
Non-invasive mapping f connections between human thalamus and cortex using diffusion imaging
Nat. Neurosci.
Towards pinpointing the neural correlates of ACT-R: a conjunction of two model-based fMRI analyses
A ballistic model of choice response time
Psychol. Rev.
The weighted likelihood ratio, linear hypotheses on normal location parameters
Ann. Math. Stat.
Cited by (137)
Multifractal characterization and recognition of animal behavior based on deep wavelet transform
2024, Pattern Recognition LettersFrom Classical Methods to Generative Models: Tackling the Unreliability of Neuroscientific Measures in Mental Health Research
2023, Biological Psychiatry: Cognitive Neuroscience and NeuroimagingNeuro-cognitive models of single-trial EEG measures describe latent effects of spatial attention during perceptual decision making
2022, Journal of Mathematical PsychologyJoint Modelling of Latent Cognitive Mechanisms Shared Across Decision-Making Domains
2024, Computational Brain and Behavior
- ☆
This work was funded by NIH award number F32GM103288. The authors would like to thank John Anderson and Michael Breakspear for insightful comments that improved an earlier version of this manuscript.