Trends in Cognitive Sciences
Volume 10, Issue 9, September 2006, Pages 424-430
Journal home page for Trends in Cognitive Sciences

Review
Beyond mind-reading: multi-voxel pattern analysis of fMRI data

https://doi.org/10.1016/j.tics.2006.07.005Get rights and content

A key challenge for cognitive neuroscience is determining how mental representations map onto patterns of neural activity. Recently, researchers have started to address this question by applying sophisticated pattern-classification algorithms to distributed (multi-voxel) patterns of functional MRI data, with the goal of decoding the information that is represented in the subject's brain at a particular point in time. This multi-voxel pattern analysis (MVPA) approach has led to several impressive feats of mind reading. More importantly, MVPA methods constitute a useful new tool for advancing our understanding of neural information processing. We review how researchers are using MVPA methods to characterize neural coding and information processing in domains ranging from visual perception to memory search.

Introduction

The most fundamental questions in cognitive neuroscience deal with the issue of representation: what information is represented in different brain structures; how is that information represented; and how is that information transformed at different stages of processing? Functional MRI (fMRI) constitutes a powerful tool for addressing these questions: While a subject performs a cognitive task, we can obtain estimates of local blood flow (a proxy for local neural processing) from tens of thousands of distinct neuroanatomical locations, within a matter of seconds. However, the large size of these datasets (up to several gigabytes) and the high levels of noise inherent in fMRI data pose a challenge to researchers interested in mining these datasets for information about cognitive processes.

Traditionally, fMRI analysis methods have focused on characterizing the relationship between cognitive variables and individual brain voxels (volumetric pixels). This approach has been tremendously productive. However, there are limits on what can be learned about cognitive states by examining voxels in isolation. The goal of this article is to describe a different approach to fMRI analysis, where — instead of focusing on individual voxels — researchers use powerful pattern-classification algorithms, applied to multi-voxel patterns of activity, to decode the information that is represented in that pattern of activity. We call this approach multi-voxel pattern analysis (MVPA).

The idea of applying multivariate methods to fMRI data (i.e. analyzing more than one voxel at once) is not new. For example, several researchers have used multivariate methods to characterize functional relationships between brain regions (e.g. 1, 2, 3, 4, 5). A major development in the last few years is the realization that fMRI data analysis can be construed, at a high level, as a pattern-classification problem (i.e. how we can recognize a pattern of brain activity as being associated with one cognitive state versus another). As such, all of the techniques that have been developed for pattern classification and data mining in other domains (e.g. handwriting recognition) can be productively applied to fMRI data analysis. This realization has led to a dramatic increase in the number of researchers using pattern-classification techniques to analyze fMRI data. This trend in the fMRI literature is part of a broader trend towards the application of pattern-classification methods in neuroscience (for applications to EEG data, see 6, 7, 8, 9, 10, 11; for applications to neural recording data from animal studies, see 12, 13, 14).

The first part of the article provides an overview of the main benefits of the MVPA approach, as well as a listing of some of the feats of ‘mind reading’ that have been accomplished with MVPA. The next part provides a more detailed overview of the methods that make this mind reading possible. The third part of the article discusses some case studies in how researchers can go beyond mind reading (for its own sake), and use MVPA to address meaningful questions about how information is represented and processed in the brain.

Section snippets

More sensitive detection of cognitive states

Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity. Conventional fMRI analysis methods try to find voxels that show a statistically significant response to the experimental conditions. To increase sensitivity to a particular condition, these methods spatially average across voxels that respond significantly to that condition. Although this approach reduces

MVPA methods

The basic MVPA method is a straightforward application of pattern classification techniques, where the patterns to be classified are (typically) vectors of voxel activity values. Figure 1 illustrates the four basic steps in an MVPA analysis. The first step, feature selection, involves deciding which voxels will be included in the classification analysis (Figure 1a); Box 1 describes feature selection in more detail. The second step, pattern assembly, involves sorting the data into discrete

MVPA case studies: going beyond mind-reading

The previous two sections focused on describing the MVPA method and how it affords increased sensitivity in detecting cognitive states. In this section, we present two case studies that show how these methodological advances are being harnessed to test theories of how the brain processes visual information 23, 24. Box 3 presents a case study of how MVPA methods are being employed to study memory retrieval [29].

Conclusions

MVPA has evolved extensively in the 5 years since the publication of the Haxby et al. [16] object categories study, and we expect that MVPA methods will continue to evolve rapidly in the coming years. A promising development in this regard is the debut, earlier this year, of an annual ‘brain activity interpretation’ competition (see http://www.ebc.pitt.edu/competition.html). This competition should facilitate the development of better algorithms for feature selection and classification by

References (54)

  • C. Davatzikos

    Classifying spatial patterns of brain activity with machine learning methods: application to lie detection

    Neuroimage

    (2005)
  • J.D. Haynes et al.

    Predicting the stream of consciousness from activity in human visual cortex

    Curr. Biol.

    (2005)
  • S. LaConte

    The evaluation of preprocessing choices in single-subject BOLD fMRI using NPAIRS performance metrics

    Neuroimage

    (2003)
  • S. LaConte

    Support vector machines for temporal classification of block design fMRI data

    Neuroimage

    (2005)
  • S. Strother

    Optimizing the fMRI data-processing pipeline using prediction and reproducibility performance metrics: I. a preliminary group analysis

    Neuroimage

    (2004)
  • J. Mourao-Miranda

    Classifying brain states and determining the discriminating activation patterns: Support vector machine on functional MRI data

    Neuroimage

    (2005)
  • K. Grill-Spector et al.

    fMR-adaptation: a tool for studying the functional properties of human cortical neurons

    Acta Psychol. (Amst.)

    (2001)
  • A.P.R. Smith

    fMRI correlates of the episodic retrieval of emotional contexts

    Neuroimage

    (2004)
  • R.C. deCharms

    Learned regulation of spatially localized brain activation using real-time fMRI

    Neuroimage

    (2004)
  • K.J. Friston et al.

    Functional connectivity

  • V.D. Calhoun

    Spatial and temporal independent component analysis of functional MRI data containing a pair of task-related waveforms

    Hum. Brain Mapp.

    (2001)
  • A. Vallabhaneni et al.

    Motor imagery task classification for brain computer interface applications using spatiotemporal principle component analysis

    Neurol. Res.

    (2004)
  • M.G. Philiastides et al.

    Temporal characterization of the neural correlates of perceptual decision making in the human brain

    Cereb. Cortex

    (2006)
  • J.M. Carmena

    Learning to control a brain-machine interface for reaching and grasping by primates

    PLoS Biol.

    (2003)
  • C.P. Hung

    Fast readout of object identity of macaque inferior temporal cortex

    Science

    (2005)
  • D.Y. Tsao

    A cortical region consisting entirely of face-selective cells

    Science

    (2006)
  • N. Kriegeskorte

    Information-based functional brain mapping

    Proc. Natl. Acad. Sci. U. S. A.

    (2006)
  • Cited by (1859)

    View all citing articles on Scopus
    View full text