Representational Neural Mapping of Dexterous Grasping Before Lifting in Humans

Summary.

Subjects lay supine in the scanner with their head, shoulders, and right upper arm securely and comfortably padded to minimize excessive motion artifact. Following a T1- anatomical scan, BOLD activity was measured as subjects were asked to reach, precision grasp, and lift an inverted T-shaped object with a CoM on the left and right, respectively (Fig. 1). The goal of the task was to minimize tilting the object. The extent to which grasp configuration and lift force varied between left and right CoM conditions was manipulated by instructing subjects to grasp the object collinearly (requiring a non-collinear force distribution) or to grasp the object non-collinearly (requiring a more symmetrical force distribution). Following preprocessing of structural and fMRI data and first-level convolution- and deconvolution-based general linear models at the individual subject and run level, vRSAs assessed the extent to which predefined regions were sensitive to grasp configuration differences or lift force differences between left and right CoM conditions over a given trial or at differing time points during the course of the trial.

Experimental design and procedure.

The experiment had four within-subject conditions (Fig. 1): manipulating an object with a (1) left and (2) right CoM at collinear grasp contacts and with a (3) left and (4) right CoM at non-collinear grasp contacts. This 2 × 2 design allowed setting up and testing model comparisons based on contrasts in which either the lift force or grasp configuration varied between left and right CoM conditions (to test the extent to which each of these contrasts contributed to a region's response pattern).

The compensatory torque (Tcom) can be generated by a combination of lift force (Ftan), grip force (Fn), and grasp configuration (GC), according to the following: where ΔFtan is the difference in lift force between the thumb and the index finger, d is the grip width (80 mm), and ΔGC is the difference between the vertical coordinate of the thumb and index finger position. In line with previous work (Marneweck et al., 2015), a pilot study on 10 healthy adult subjects doing the same task with a similar object outside the scanner showed no difference in mean grip force between the left and right CoM conditions (p's > 0.05). This suggests that the difference in lift force and grasp configuration of the thumb and index finger would be the predominant means of generating an appropriate torque (thereby achieving task success) on objects with left- and right-sided CoMs. In a previous study (Marneweck et al., 2018), we found no differences in grasp kinematics when subjects in a supine position manipulated the object inside the scanner compared with sitting up outside the scanner. Thus, we did not foresee differences in grasp kinematics while subjects grasped the object collinearly or non-collinearly and applying the appropriate lift forces while lying down (inside the scanner) in the current study.

In conditions 1 and 2, the grasp configuration of the thumb and index finger were constrained to be collinear (ΔGC = 0). An appropriate moment to counteract the external torque of the object with a collinear grasp configuration requires an asymmetric lift force distribution (more lift force by the thumb than index finger in the left CoM condition and vice versa in the right CoM condition). Thus, the lift force distribution varies between condition 1 and 2 while the grasp configuration is unchanged. In conditions 3 and 4, the grasp configuration of the thumb and index finger were constrained to be non-collinear, with the digit closest to the CoM on the upper grasp surface and the other digit on the lower grasp surface. With the vertical distance between the pair of grasp surfaces set at 1.60 cm, and assuming a mean grip force of 14 N (SD = 3.50) on an object of similar weight and torque (based on our pilot data), this digit positioning difference should equate to a more symmetrical lift force distribution by each digit on a successful trial (and thus less difference in digit lift force between left and right CoM conditions than left and right CoM conditions at collinear grasp contacts). Thus, in conditions 3 and 4, subjects were to use a non-collinear digit position partitioning (with the thumb higher than the index finger when the CoM is on the left, and vice versa when the CoM is on the right) in combination with more similar digit lift forces for generating an appropriate compensatory torque. Critically, in all four conditions, subjects were to modify and integrate load force distribution based on the specified grasp configuration to achieve an appropriate compensatory torque.

Subjects first performed a block of 32 familiarization trials in which they lifted an object where its CoM alternated between left and right every four trials. Following this, subjects performed six functional runs of 28 trials (seven blocked trials of each of the four conditions), with an intertrial interval randomly chosen to be 2, 3, 4, 5, or 6 s, with a rest period between each of the four conditions (during which time the experimenter changed the CoM). The duration of the rest periods was ∼30 s (depending on the time it took the experimenter to change the CoM and provide the instruction for the upcoming block of trials). The order of the blocks of collinear and non-collinear grasp trials and the order of the blocked CoM within each of those trials were counterbalanced across runs and subjects. Subjects were informed about the CoM and digit position condition before the start of each of the four blocks within a given run. On each trial, subjects pressed the button with their right hand in a relaxed position until an audio start tone instructed them to reach, grasp and lift the object, hold it at the height of a marker (4 cm) until an audio stop tone (4 s after the start tone), after which the object was to be placed in its original position and the hand returned to the button. After this, an error tone was played if the object rolled >5° in either direction during the trial. The start tone of the first trial in each functional run always coincided with the beginning of a new functional image.

Anatomical and functional MRI data were acquired using a Siemens 3T Magnetom Prisma Fit (64-channel phased-array head coil). High-resolution 0.94 mm isotropic T1-MPRAGE (TR = 2500 ms, TE = 2.2 ms, FA = 7°, FOV = 241 mm) sagittal sequence images were acquired of the whole brain. Following this, subjects performed the object manipulation task during which BOLD contrast was measured with a CMRR multiband (University of Minnesota) T2*-weighted echo planar gradient-echo imaging sequence (TR = 400 ms, TE = 35 ms, FA = 52°, FOV = 192 mm, multiband factor 8). Each functional image consisted of 48 slices acquired parallel to the AC-PC plane (3 mm thick; 3 × 3 mm in-plane resolution). The behavioral experimental protocol was interleaved between 25 TRs at the start and at the end of each functional run.

Kinematic data processing and analyses.

Data collected were filtered using a fourth order Butterworth filter with a cutoff frequency of 5 Hz. Reach onset was defined as the button release onset. Object lift onset was defined as the time point at which the vertical position of the object went >1 mm and remained above this value for 20 samples. The end of the lift was defined as the time at which the hand was back at the button. A full trial was classified as the duration from reach onset to the end of the lift. Object roll was defined as the angle of the object in the oblique plane. Peak object roll was recorded shortly after lift onset (∼250 ms) before somatosensory feedback resulted in corrected feedback responses to counter object roll. Trials with object roll > 5° were classified as errors. Error trials were not of interest and were not analyzed, but they were modeled as a separate regressor in the general linear model (GLM) of each run for each subject. We were only interested in successful trials, which depended on the subject having the appropriate sensorimotor memory to generate an appropriate compensatory moment needed to minimize tilt (since feedback of object properties only became available after lift onset). Therefore, any carry-over adaptation effects to the next CoM block of trials were not of interest to the current research question and excluded from the analyses. Nevertheless, these carry-over adaptation effects were minimal with a very small proportion of trials classified as errors (M = 0.029, SD = 0.027).

MRI data processing and analyses.

MRI data were preprocessed and analyzed using SPM12 (Wellcome Trust Center for Neuroimaging, London, UK) and FSL (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/; Jenkinson et al., 2012). Each subject's functional images from all runs were spatially realigned to a mean image using second degree B-spline interpolation, and then coregistered to the T1 using SPM. Between-subject normalization of the cerebellum was conducted using the SUIT SPM toolbox (Diedrichsen, 2006; Diedrichsen et al., 2009, 2011; Diedrichsen and Zotow, 2015) (interpolation: trilinear, voxel size: 2×2×2 mm) and between-subject normalization of the rest of the brain was conducted with SPM's normalize function.

A Bayesian implementation of RSA, vRSA (Friston et al., 2019) was applied using an adaptation of the MATLAB script DEMO_CVA_RSA.m available in SPM12. This was used to assess the evidence for condition-specific patterns of responses distributed over voxels in a set of predefined regions. In an initial preparatory step, we extracted regional multivoxel patterns for each of the conditions of interest. This was done two ways: by performing both a convolution- and deconvolution-based GLM for each run and for each subject. In the convolution method, a GLM with events convolved with the standard canonical HRF basis function in SPM was estimated, with four experimental conditions (left CoM collinear grasp, left CoM non-collinear grasp, right CoM collinear grasp, right CoM non-collinear grasp) and an error condition (if any occurred in a run) entered as regressors. The rationale for this GLM was to identify those regions that were sensitive by RSA analysis to any of the tested contrasts regardless of when the distinctive patterns occurred across the whole trial. In addition, we used the opportunity to replicate findings from a previous study where we used an RSA approach based on crossnobis distances to identify neural representations encoding differences in torque direction (by contrasting left and right-weighted object interactions) (Marneweck et al., 2018). Thus, the whole trial was modeled for each event, from reach onset (button release) to the end of the lift (button down). For regions showing sensitivity to any of the main effects or interaction effects as tested by vRSA (see below), a subsequent deconvolution-based GLM approach, with a finite impulse response (FIR) function for each run and subject was computed, with the onsets for each of the four experimental conditions set to lift onset (window length: 7.2 s; order: 800 ms). This time window and order was selected to sufficiently track activations through the peak of the HRF, which was assumed to occur ∼6 s after lift onset. The rationale for deriving β values from a deconvolution-based GLM approach was to use vRSA to assess our hypothesis that condition-specific responses might occur in a temporally distinct manner (i.e., first grasp configuration, then lift force). In both convolution and deconvolution-based GLMs, the RobustWLS Toolbox in SPM (Diedrichsen and Shadmehr, 2005) was used to down-weight functional images with high noise variance to account for movement artifact.

A second preparatory step to vRSA was the extraction of the GLM-derived β values for each of the four conditions from regions of interest (ROIs) using FSL's fslmeants. β values from ROIs were extracted separately for each run. Predefined cerebellar and cortical ROIs were selected that have previously been shown to be sensitive to differences when manipulating objects of different torques (Marneweck et al., 2018). Left-hemisphere cortical ROIs (Fig. 2A) were extracted from the SPM Anatomy Toolbox (Eickhoff et al., 2005, 2006, 2007) including motor area 4a, motor area 4p, anterior intraparietal area (AIP), superior parietal area 5 and 7 (SPL5, SPL7), primary central sulcus (PSC/SI), parietal operculum area 1 and 4 (OP1, OP4), Broca area (BA) 44, inferior parietal area (IPL), lateral occipital cortex (LOC), and a control region, auditory cortex (AUD). Dorsal and ventral premotor areas (PMd and PMv) and supplementary motor area (SMA) were free-drawn on a standardized surface mesh in SUMA (Saad et al., 2004) based on predefined anatomical parcellations (Geyer et al., 1996; Picard and Strick, 2001; Tomassini et al., 2007; Destrieux et al., 2010), which were projected to standard MNI space and mapped backed to the subject's T1-weighted image (Barany et al., 2014). Superior parieto-occipital cortex (SPOC) was defined according to a previous functional study (Fabbri et al., 2012) as per (Di Bono et al., 2015) (centered MNI coordinates: −17, −76, 40; diameter: 8 mm). Cerebellar ROIs (Fig. 2B) were extracted from a recently published cerebellar functional atlas (King et al., 2019), which activated during right-hand movements (region 2) and motor planning (region 2 and region 4). β values from a cerebellar control region that activated emotional and language processing (region 7) were also extracted.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

A, Cortical and (B) cerebellar regions of interest (ROI) displayed on the MNI-152 atlas using the visualization software, Surf Ice (https://www.nitrc.org/projects/surfice/). The center of gravity for each cortical ROI cluster is depicted here rather than the whole cluster given substantial between-ROI cortical surface overlap. All cortical regions (including those on the medial wall) are projected to the lateral surface.

Similar to a more traditional RSA approach, vRSA contrasts between-condition differences in spatial voxel patterns. Whereas these between-condition contrasts are specified in terms of representational dissimilarity matrix in traditional RSA, in vRSA, the contribution of between-condition contrasts to a region's response pattern is expressed in terms of second-order similarity or covariance matrices. These matrices summarize the relationship between patterns in terms of each of the experimental conditions. vRSA was chosen over traditional RSA for its robustness in identifying the consistency of a given contribution across multiple runs and also at the between-subject level. As with other pattern component modeling methods, it provides a formal method for testing the effect on the contribution of more than one contrast to a region's response pattern while taking into account all specified contrasts and their interactions in a model comparison framework. Specifically, vRSA uses Bayesian model comparisons to assess the evidence for a region to be sensitive to different contrasts by comparing the log evidence between the group level model contrast and that same model contrast when one hyperparameter of another model contrast is decreased toward zero by specifying precise hyperpriors, thereby essentially removing its contribution. A relative log evidence of 3 corresponds to a Bayes factor of approximately exp(3) ≈ 20 to 1. The Bayes factor is a fundamental part of the Bayesian approach to testing hypotheses, providing a continuous degree or measure of evidence for the null and alternative hypotheses, H0 and H1 (see Dienes and Mclatchie, 2018, for a recent review). It also provides evidence for competing models as is done here, which can provide evidence that a region is sensitive to more than one contrast. With a Bayes factor of 1, both H0 and H1 models predict the data equally well and the evidence does not favor either model over the other. The evidence favors H1 over H0 when the Bayes factor increases beyond 1 (toward infinity). The evidence favors H0 over H1 when the Bayes factor decreases <1 (toward 0). Dissimilar to fixed significance or threshold levels of the Neyman-Pearson frequentist approach, this Bayesian approach offers a continuous degree of evidence. Nonetheless, rough guidelines have been provided similar to that of Cohen (1988) for effect sizes. Particularly, Jeffreys (1998) and Kass and Raftery (1995) have suggested that a Bayes factor of approximately 3 matches a “substantial” amount of evidence that a contrast of interest contributes to a region's observed response pattern. Moreover, Dienes (2014) argued a Bayes factor of approximately 3 occurs when a result is just significant using the frequentist approach. As will be seen below, our log evidence values were well above this criterion value.

As outlined in Friston et al. (2019) technical note describing variational implementation of representational similarity analyses (vRSA), a crucial step in this analysis is the introduction of a prior on the hyperparameters (also known as hyperpriors). For this study the following priors were used within the vRSA script: hyperprior expectation in log-space = −32; hyperprior covariance in log-space = 256; and prior expectation and covariance of reduced model = 1/256. The hyperprior expectation value represents the prior belief in the strength of each of the tested components e.g., exp(−32) = 1.2664e-14. That is, each component is something positive that is really close to zero. The larger this value, the stronger the belief in the tested component. The second parameter, the hyperprior covariance, is the prior belief about the covariance between the tested components. For example, if a region has a pattern for a CoM effect, what is our belief it also has a pattern for a grasp configuration effect or their interaction. The larger the number, the stronger the belief that they could covary, making it easier to get multiple components significant. We selected the default hyperprior values from the spm_reml_sc.m script (−32, 256) based on the lack of research on how the brain represents grasping and lifting in a coordinated way. In other words, we had little evidence or prior beliefs as to how the brain might represent these behaviors and therefore adopted a conservative uninformed approach in the selection of hyperpriors.

With the present study's 2 × 2 design (CoM × grasp), the contribution of three contrasts to ROI response patterns were assessed using vRSA. The first, testing a main effect CoM, contrasted conditions in which the object was lifted with a left and right CoM (regardless of non-collinear and collinear grasp contact points). The second, testing a main effect of grasp, contrasted conditions in which the object was lifted at non-collinear and collinear grasp contact points (regardless of the CoM). The third, testing an CoM × grasp interaction effect, contrasted left and right-CoM conditions at collinear and non-collinear grasp contact points, respectively. A significant interaction would suggest that the region is either sensitive to a left versus right CoM difference at non-collinear or collinear grasp contact points. As will be detailed below, vRSAs on the convolution-modeled data for the most part showed that ROIs sensitive to a CoM effect were also sensitive to a grasp effect, and interactions were rare. This suggested that regions encoded both torque direction, grasping and/or lifting (in large part independently) and with a convolution-based GLM it was difficult to ascertain the extent to which these processes were encoded in a temporally distinct manner. To this end, a deconvolution-based vRSA approach explored the possibility that these processes are contributing to a region's response at differing time courses. Furthermore, regions showing interaction effects would suggest the region is either sensitive to grasp configuration differences between left and right CoM conditions or lift force differences between left and right CoM conditions. To further explore these, the contribution of two additional contrasts to a given ROI's response pattern was tested using vRSA on deconvolution-modeled BOLD activity: non-collinear left CoM versus non-collinear right CoM (manipulating grasp configuration) and colinear left CoM versus collinear right CoM (manipulating lift force).

Critically, the Bayes criterion assessing the merits of the different models (i.e., across different FIRs and contrasts) are all derived from the same hierarchical Bayes estimation. There are no repeated estimations in this context. In hierarchical Bayesian analyses that incorporates the between-subject consistency of a given effect/component as we have performed, the interpretation of the data is not influenced by multiple comparisons (Gelman et al., 2012; Kruschke, 2015; Gelman and Loken, 2016). Finally, Friston et al. (2019) details how the vRSA approach accounts for covariance within a region of p voxels. This covariance is estimated in the hierarchical Bayes model (equation 7 in Friston et al., 2019). This accounts for ROIs with different numbers of voxels and covariance between voxels, analogous to crossnobis distances in RSA. As a secondary check, we correlated ROI sizes with log evidence values for each of the components from the first vRSA and found no correlation between ROI size and log evidence values (CoM: r = 0.21, grasp: r = −0.01; interaction: r = −0.18).