Finding decodable information that can be read out in behaviour

Highlights

• We tested whether decodable information can be used by the brain for behaviour.

• Only a subset of decodable representations predicted behaviour.

• This has important implications for the interpretation of neuroimaging studies.

• The results highlight the importance of relating decoding results to behaviour.

Abstract

Multivariate decoding methods applied to neuroimaging data have become the standard in cognitive neuroscience for unravelling statistical dependencies between brain activation patterns and experimental conditions. The current challenge is to demonstrate that decodable information is in fact used by the brain itself to guide behaviour. Here we demonstrate a promising approach to do so in the context of neural activation during object perception and categorisation behaviour. We first localised decodable information about visual objects in the human brain using a multivariate decoding analysis and a spatially-unbiased searchlight approach. We then related brain activation patterns to behaviour by testing whether the classifier used for decoding can be used to predict behaviour. We show that while there is decodable information about visual category throughout the visual brain, only a subset of those representations predicted categorisation behaviour, which were strongest in anterior ventral temporal cortex. Our results have important implications for the interpretation of neuroimaging studies, highlight the importance of relating decoding results to behaviour, and suggest a suitable methodology towards this aim.

Introduction

Multivariate pattern analysis (MVPA), also called brain decoding, is a powerful tool to establish statistical dependencies between experimental conditions and brain activation patterns (Carlson et al., 2003; Cox and Savoy, 2003; Haxby et al., 2001; Haynes, 2015; Kamitani and Tong, 2005; Kriegeskorte et al., 2006). In these analyses, an implicit assumption often made by experimenters is that if information can be decoded, then this information is used by the brain in behaviour (de-Wit et al., 2016; Ritchie et al., 2017). However, the decoded information could be different (e.g., epiphenomenal) from the signal that is relevant for the brain to use in behaviour (de-Wit et al., 2016; Williams et al., 2007), highlighting the need to relate decoded information to behaviour. Importantly, this implicit assumption of decoding models leads to testable predictions about task performance (Naselaris et al., 2011). Previous work has for example correlated decoding performances to behavioural accuracies (Bouton et al., 2018; Freud et al., 2017; Raizada et al., 2010; van Bergen et al., 2015; Walther et al., 2009; Williams et al., 2007). However, this does not model how individual experimental conditions relate to behaviour. Another approach has been to compare neural and behavioural similarity structures (Bracci and Op de Beeck, 2016; Cichy et al., 2017; Cohen et al., 2016; Grootswagers et al., 2017a; Haushofer et al., 2008; Mur et al., 2013; Proklova et al., 2016; Wardle et al., 2016). While this approach allows to link behaviour and brain patterns at the level of single experimental conditions, it is unclear how this link carries over to decision making behaviour such as categorisation (but see Cichy et al. (2017) for recent developments).

Recently, a novel methodological approach, called the distance-to-bound approach (Ritchie and Carlson, 2016), has been proposed to connect brain activity directly to perceptual decision-making behaviour at the level of individual experimental conditions. The rationale behind this approach (Bouton et al., 2018; Carlson et al., 2014; Kiani et al., 2014; Philiastides and Sajda, 2006; Ritchie and Carlson, 2016) is that for decision-making tasks, the brain applies a decision boundary to a neural activation space (DiCarlo and Cox, 2007). Similarly, MVPA classifiers fit multi-dimensional hyperplanes to separate a neural activation space. In classic signal-detection theory (Green and Swets, 1966) and evidence-accumulation models of choice behaviour (Brown and Heathcote, 2008; Gold and Shadlen, 2007; Ratcliff and Rouder, 1998; Smith and Ratcliff, 2004), the distance of the input to a decision boundary reflects the ambiguity of the evidence for the decision (Green and Swets, 1966). Decision evidence, in turn, predicts choice behaviour (e.g., Ashby, 2000; Ashby and Maddox, 1994; Britten et al., 1996; Gold and Shadlen, 2007; Shadlen and Kiani, 2013) which also has clear neural correlates (e.g., Britten et al., 1996; Ratcliff et al., 2009; Roitman and Shadlen, 2002). If for a decision task (e.g., categorisation), the brain uses the same information as the MVPA classifier, then the classifier's hyperplane reflects the brain's decision boundary. This in turn predicts that distance to the classifier's hyperplane negatively correlates with reaction times for the decision task. In the distance-to-bound approach, finding such a negative distance-RT-correlation shows that the information is then suitably formatted to guide behaviour. “Suitably formatted to guide behaviour” here means that the information is structured in such a way that the brain can apply a linear read out process to this representation to make a decision (importantly, this does not imply a causal link with behaviour). Carlson et al. (2014) demonstrated the promise of the distance-to-bound approach in a region of interest based analysis using fMRI. Here we go beyond this work by using the distance-to-bound method and a spatially unbiased fMRI-searchlight approach to create maps of where in the brain information can be used to guide behaviour.