Elsevier

NeuroImage

Volume 72, 15 May 2013, Pages 193-206
NeuroImage

A Bayesian framework for simultaneously modeling neural and behavioral data

https://doi.org/10.1016/j.neuroimage.2013.01.048Get rights and content

Abstract

Scientists who study cognition infer underlying processes either by observing behavior (e.g., response times, percentage correct) or by observing neural activity (e.g., the BOLD response). These two types of observations have traditionally supported two separate lines of study. The first is led by cognitive modelers, who rely on behavior alone to support their computational theories. The second is led by cognitive neuroimagers, who rely on statistical models to link patterns of neural activity to experimental manipulations, often without any attempt to make a direct connection to an explicit computational theory. Here we present a flexible Bayesian framework for combining neural and cognitive models. Joining neuroimaging and computational modeling in a single hierarchical framework allows the neural data to influence the parameters of the cognitive model and allows behavioral data, even in the absence of neural data, to constrain the neural model. Critically, our Bayesian approach can reveal interactions between behavioral and neural parameters, and hence between neural activity and cognitive mechanisms. We demonstrate the utility of our approach with applications to simulated fMRI data with a recognition model and to diffusion-weighted imaging data with a response time model of perceptual choice.

Highlights

► We present a method for combining behavioral and neural data into a single analysis. ► The method uses neural data to constrain behavioral models. ► Our method allows for any behavioral model to be combined with any neural model.

Introduction

Currently, there are two main methods to study cognition. The first and oldest method is known as cognitive modeling. Given a set of experimental data, one assumes that observers use a particular process, known as a cognitive model, to produce the observed data. The processes used by the cognitive model are controlled by a set of unknown parameters. The parameters of the cognitive model are then estimated and psychologically meaningful interpretations are based on these parameter estimates. While cognitive models have been effective tools for identifying how cognition changes as a function of task demands, they suffer from being highly abstract representations of what is essentially a system of biological processes. The second method to study cognition is to make contact with the biological substrate more directly and measure brain activity using methods such as positron emission tomography (PET), functional-magnetic resonance imaging (fMRI), electroencephalography (EEG), or diffusion-weighted imaging (DWI), and we will refer to this broad class of data as “neural data”. While neural data provide valuable information about the biological and physical aspects of cognition, traditional neural imaging analyses (e.g., general linear models) are limited because they do not attempt to describe cognitive processes. Because both methods for studying cognition have clear advantages and disadvantages (Wilkinson and Halligan, 2004), there has been a recent surge of interest in combining both sources of information to provide a single explanation of the underlying process (e.g., Anderson et al., 2008, Borst et al., 2011, Dolan, 2008, Forstmann et al., 2008, Forstmann et al., 2010, Forstmann et al., 2011, Gläscher and O'Doherty, 2010, O'Doherty et al., 2007).

In this article, we propose a general framework for describing neural and behavioral data with a single model. Our approach is to treat the two sources of information as separate measurements of the same cognitive construct. To fit the model, we make use of a hierarchical Bayesian approach, which has become an important method for inference in both the neural (e.g., Friston et al., 2002, Gershman et al., 2011, Guo et al., 2008, Wu et al., 2011), and cognitive modeling (e.g., Lee, 2011, Shiffrin et al., 2008) literatures.

Using the hierarchical Bayesian approach provides a number of benefits. First, the Bayesian framework provides meaningful, interpretable information at both the subject and group levels. Second, the Bayesian framework lends itself naturally to principled inclusion of missing data. We will show how our framework allows us to make predictions for missing data, based solely on parameter relationships learned from fitting the model. In particular, we show that we can make informed predictions of behavioral data given only neural data, and vice versa. Third, our framework allows us to infer relationships between parameters, relationships that need not be hypothesized a priori. This feature affords us explorative opportunities in the form of the Bayesian posterior distribution. Fourth, the framework we propose does not require a commitment to any particular model, as in other joint modeling approaches (e.g., Anderson et al., 2008, Borst et al., 2011, Mazurek et al., 2003). By using a hierarchical Bayesian approach, we can choose any particular cognitive model to explain the behavioral data, and any neural model to explain the neural data. Subsequently, our framework links the two models together and simultaneously infers meaningful relationships between the two models while also providing a unifying account of brain and behavioral data.

Using such a framework also provides a method for answering much more general questions, which we do not attempt to answer here. For example, linking brain and behavioral data allows us to directly perform model selection on multiple theories of cognition. One could fit several different cognitive models combined with a single neural model to data, and the joint model that fit the full data set best would be the preferred model. In this way, the neural data provides deeper constraints on cognitive models, and in so doing, can be used to better test cognitive theories.

We first provide a brief introduction to the two different types of measurements, and then describe our joint modeling approach. We then demonstrate the utility of our approach in a simulation study. Finally, we apply our model to data from an experiment containing both neural data and behavioral data that can be fit with a computational model. We show that meaningful relationships between model parameters and neural data can be inferred directly from fitting the model, and these relationships can be further exploited to make predictions about the distribution of missing or unobserved data.

Section snippets

Prior research

Although both behavioral and neural data are central to the study of cognition, few attempts have been made to merge them. Perhaps one of the most successful approaches toward this goal is model-based fMRI analysis (Gläscher and O'Doherty, 2010, O'Doherty et al., 2007). In this procedure, a cognitive model is first used to simulate neural data. To do this, often cognitive models are convolved with particular functions that resemble neural effects, such as the hemodynamic response function that

The framework

We wish to provide a joint explanation for the jth subject's neural Nj and behavioral Bj data. If it is difficult or undesirable to specify the joint distribution of (Nj, Bj) under a single model, we can begin by describing how each individual source should be modeled. We will denote the cognitive model as Behav with unknown parameters θ, and the neural model as Neural with unknown parameters δ. A key benefit of our framework is that we are not limited to a particular cognitive or neural model.

Simulation study

In order to highlight the advantages of our approach we conducted a simulation study in which we generated data from the joint model so that the neural side (i.e., the left side of Fig. 1) consisted of fMRI scans and the behavioral side (i.e., the right side of Fig. 1) consisted of data from a recognition memory task. The simulation was designed to mimic a typical recognition memory experiment in which, during a study phase, a subject is provided with a single set of items (e.g., words or

Fitting the joint model to response time and tract strength data

In this section, we will demonstrate the joint model's ability to generalize and predict future data for new subjects while having only neural data or some combination of neural data and (sparse) behavioral data. We demonstrate this feature of the model on experimental data reported in Forstmann et al. (2011). The study was designed to provide further evidence for the striatal hypothesis of the speed accuracy tradeoff (Bogacz et al., 2010, Forstmann et al., 2008, Forstmann et al., 2010), which

General discussion

We have presented a new approach to unifying neural and cognitive models to better understand cognition. In contrast to previous methods, our method is extremely flexible because it can combine any neural model with any cognitive model. Our method also provides a way to infer the relationships that might exist between the biological and physical properties of the brain and higher-level cognitive processes.

By combining two separate models into a single Bayesian hierarchical model, we can make

Conclusions

In this article, we have presented a hierarchical Bayesian framework for combining both neural and cognitive models into a single unifying model. We have shown that our approach provides a number of benefits over current approaches and allows for principled inference of the relationships between the biological properties of the brain assessed by neuroimaging techniques and the theories of cognition that are used to understand higher levels of cognitive processing. With this approach, one can

References (62)

  • K. Friston et al.

    Classical and Bayesian inference in neuroimaging

    NeuroImage

    (2002)
  • K. Friston et al.

    Dynamic causal modeling

    NeuroImage

    (2003)
  • S.J. Gershman et al.

    A tutorial on Bayesian nonparametric models

    J. Math. Psychol.

    (2012)
  • S.J. Gershman et al.

    A topographic latent source model for fMRI data

    NeuroImage

    (2011)
  • C.J.H. Ludwig et al.

    The mechanism underlying inhibition of saccadic return

    Cogn. Psychol.

    (2009)
  • J.W. Mink

    The basal ganglia: focused selection and inhibition of competing motor programs

    Prog. Neurobiol.

    (1996)
  • J.A. Mumford et al.

    Deconvolving bold activation in event-related designs for multivoxel pattern classification analyses

    NeuroImage

    (2012)
  • D.J. Navarro et al.

    Modeling individual differences using Dirichlet processes

    J. Math. Psychol.

    (2006)
  • J.P. O'Doherty et al.

    Temporal difference models and reward-related learning in the human brain

    Neuron

    (2003)
  • A. Quirós et al.

    Bayesian spatiotemporal model of fMRI data

    NeuroImage

    (2010)
  • B.M. Turner et al.

    Approximate Bayesian computation with differential evolution

    J. Math. Psychol.

    (2012)
  • M.A.J. Van Gerven et al.

    Efficient Bayesian multivariate fMRI analysis using a sparsifying spatio-temporal prior

    NeuroImage

    (2010)
  • R. Wetzels et al.

    An encompassing prior generalization of the Savage–Dickey density ratio

    Comput. Stat. Data Anal.

    (2010)
  • W. Wu et al.

    A hierarchical Bayesian approach for learning sparse spatio-temporal decompositions of multichannel EEG

    NeuroImage

    (2011)
  • J.R. Anderson

    How Can the Human Mind Occur in the Physical Universe?

    (2007)
  • J.R. Anderson et al.

    Using fMRI to test models of complex cognition

    Cogn. Sci.

    (2008)
  • J.R. Anderson et al.

    Neural imaging to track mental states

    Proc. Natl. Acad. Sci. U. S. A.

    (2010)
  • T. Behrens et al.

    Non-invasive mapping f connections between human thalamus and cortex using diffusion imaging

    Nat. Neurosci.

    (2003)
  • J. Borst et al.

    Towards pinpointing the neural correlates of ACT-R: a conjunction of two model-based fMRI analyses

  • S. Brown et al.

    A ballistic model of choice response time

    Psychol. Rev.

    (2005)
  • J.M. Dickey

    The weighted likelihood ratio, linear hypotheses on normal location parameters

    Ann. Math. Stat.

    (1971)
  • Cited by (137)

    View all citing articles on Scopus

    This work was funded by NIH award number F32GM103288. The authors would like to thank John Anderson and Michael Breakspear for insightful comments that improved an earlier version of this manuscript.

    View full text