Trends in Cognitive Sciences
ReviewBeyond mind-reading: multi-voxel pattern analysis of fMRI data
Introduction
The most fundamental questions in cognitive neuroscience deal with the issue of representation: what information is represented in different brain structures; how is that information represented; and how is that information transformed at different stages of processing? Functional MRI (fMRI) constitutes a powerful tool for addressing these questions: While a subject performs a cognitive task, we can obtain estimates of local blood flow (a proxy for local neural processing) from tens of thousands of distinct neuroanatomical locations, within a matter of seconds. However, the large size of these datasets (up to several gigabytes) and the high levels of noise inherent in fMRI data pose a challenge to researchers interested in mining these datasets for information about cognitive processes.
Traditionally, fMRI analysis methods have focused on characterizing the relationship between cognitive variables and individual brain voxels (volumetric pixels). This approach has been tremendously productive. However, there are limits on what can be learned about cognitive states by examining voxels in isolation. The goal of this article is to describe a different approach to fMRI analysis, where — instead of focusing on individual voxels — researchers use powerful pattern-classification algorithms, applied to multi-voxel patterns of activity, to decode the information that is represented in that pattern of activity. We call this approach multi-voxel pattern analysis (MVPA).
The idea of applying multivariate methods to fMRI data (i.e. analyzing more than one voxel at once) is not new. For example, several researchers have used multivariate methods to characterize functional relationships between brain regions (e.g. 1, 2, 3, 4, 5). A major development in the last few years is the realization that fMRI data analysis can be construed, at a high level, as a pattern-classification problem (i.e. how we can recognize a pattern of brain activity as being associated with one cognitive state versus another). As such, all of the techniques that have been developed for pattern classification and data mining in other domains (e.g. handwriting recognition) can be productively applied to fMRI data analysis. This realization has led to a dramatic increase in the number of researchers using pattern-classification techniques to analyze fMRI data. This trend in the fMRI literature is part of a broader trend towards the application of pattern-classification methods in neuroscience (for applications to EEG data, see 6, 7, 8, 9, 10, 11; for applications to neural recording data from animal studies, see 12, 13, 14).
The first part of the article provides an overview of the main benefits of the MVPA approach, as well as a listing of some of the feats of ‘mind reading’ that have been accomplished with MVPA. The next part provides a more detailed overview of the methods that make this mind reading possible. The third part of the article discusses some case studies in how researchers can go beyond mind reading (for its own sake), and use MVPA to address meaningful questions about how information is represented and processed in the brain.
Section snippets
More sensitive detection of cognitive states
Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity. Conventional fMRI analysis methods try to find voxels that show a statistically significant response to the experimental conditions. To increase sensitivity to a particular condition, these methods spatially average across voxels that respond significantly to that condition. Although this approach reduces
MVPA methods
The basic MVPA method is a straightforward application of pattern classification techniques, where the patterns to be classified are (typically) vectors of voxel activity values. Figure 1 illustrates the four basic steps in an MVPA analysis. The first step, feature selection, involves deciding which voxels will be included in the classification analysis (Figure 1a); Box 1 describes feature selection in more detail. The second step, pattern assembly, involves sorting the data into discrete
MVPA case studies: going beyond mind-reading
The previous two sections focused on describing the MVPA method and how it affords increased sensitivity in detecting cognitive states. In this section, we present two case studies that show how these methodological advances are being harnessed to test theories of how the brain processes visual information 23, 24. Box 3 presents a case study of how MVPA methods are being employed to study memory retrieval [29].
Conclusions
MVPA has evolved extensively in the 5 years since the publication of the Haxby et al. [16] object categories study, and we expect that MVPA methods will continue to evolve rapidly in the coming years. A promising development in this regard is the debut, earlier this year, of an annual ‘brain activity interpretation’ competition (see http://www.ebc.pitt.edu/competition.html). This competition should facilitate the development of better algorithms for feature selection and classification by
References (54)
Dynamic causal modelling
Neuroimage
(2003)Spatial pattern analysis of functional brain images using partial least squares
Neuroimage
(1996)- et al.
Partial least squares analysis of neuroimaging data: applications and advances
Neuroimage
(2004) Mining multi-channel EEG for its information content: an ANN-based method for a brain-computer interface
Neural Netw.
(1998)Linear spatial integration for single-trial detection in encephalography
Neuroimage
(2002)EEG-based neuroprosthesis control: a step towards clinical practice
Neurosci. Lett.
(2005)Classifying EEG-based motor imagery tasks by means of time-frequency synthesized spatial patterns
Clin. Neurophysiol.
(2004)- et al.
How distributed is visual category information in human occipito-temporal cortex? An fMRI study
Neuron
(2002) - et al.
Functional magnetic resonance imaging (fMRI) ‘brain reading’: detecting and classifying distributed patterns of fMRI activity in human visual cortex
Neuroimage
(2003) Combinatorial codes in ventral temporal lobe for object recognition: Haxby (2001) revisited: is there a ‘face’ area?
Neuroimage
(2004)
Classifying spatial patterns of brain activity with machine learning methods: application to lie detection
Neuroimage
Predicting the stream of consciousness from activity in human visual cortex
Curr. Biol.
The evaluation of preprocessing choices in single-subject BOLD fMRI using NPAIRS performance metrics
Neuroimage
Support vector machines for temporal classification of block design fMRI data
Neuroimage
Optimizing the fMRI data-processing pipeline using prediction and reproducibility performance metrics: I. a preliminary group analysis
Neuroimage
Classifying brain states and determining the discriminating activation patterns: Support vector machine on functional MRI data
Neuroimage
fMR-adaptation: a tool for studying the functional properties of human cortical neurons
Acta Psychol. (Amst.)
fMRI correlates of the episodic retrieval of emotional contexts
Neuroimage
Learned regulation of spatially localized brain activation using real-time fMRI
Neuroimage
Functional connectivity
Spatial and temporal independent component analysis of functional MRI data containing a pair of task-related waveforms
Hum. Brain Mapp.
Motor imagery task classification for brain computer interface applications using spatiotemporal principle component analysis
Neurol. Res.
Temporal characterization of the neural correlates of perceptual decision making in the human brain
Cereb. Cortex
Learning to control a brain-machine interface for reaching and grasping by primates
PLoS Biol.
Fast readout of object identity of macaque inferior temporal cortex
Science
A cortical region consisting entirely of face-selective cells
Science
Information-based functional brain mapping
Proc. Natl. Acad. Sci. U. S. A.
Cited by (1859)
Neural correlates of altered emotional responsivity to infant stimuli in mothers who use substances
2024, Journal of Psychiatric ResearchScene-selectivity in CA1/subicular complex: Multivoxel pattern analysis at 7T
2024, NeuropsychologiaNeural activity related to productive vocabulary knowledge effects during second language comprehension
2024, Neuroscience ResearchDecoding fMRI data with support vector machines and deep neural networks
2024, Journal of Neuroscience Methods