Abstract
Preparation for an action, such as grasping an object, is accompanied by an enhanced perception of the object's action-relevant features, such as orientation and size. Cortical feedback from motor planning areas to early visual areas may drive this enhanced perception. To examine whether action preparation modulates activity in early human visual cortex, subjects grasped or pointed to oriented objects while high-resolution fMRI data were acquired. Using multivoxel pattern analysis techniques, we could decode with >70% accuracy whether a grasping or pointing action was prepared from signals in visual cortex as early as V1. These signals in early visual cortex were observed even when actions were only prepared but not executed. Anterior parietal cortex, on the other hand, showed clearest modulation for actual movements. This demonstrates that preparation of actions, even without execution, modulates relevant neuronal populations in early visual areas.
Introduction
When planning to pick up an object, our brain needs to obtain accurate information about the location, size, and orientation of the target object. It is therefore essential that such relevant visual properties be examined with the maximum possible accuracy before initiating a grasping action. Enhanced perception is thus triggered by the mere intention to grasp. Several behavioral studies have shown evidence for an “action-modulated perception” mechanism that automatically enhances relevant features during action preparation (Craighero et al., 1999; Bekkering and Neggers, 2002; Hannus et al., 2005; Fagioli et al., 2007), a moment when an action is just intended but not executed yet. Recently we showed that orientation perception, a relevant feature, is enhanced during preparation of a grasping action, compared with a pointing action, for which object orientation is not important (Gutteling et al., 2011, 2013).
Improved object feature perception shortly before action performance could be mediated by feedback connections between cortical motor control areas and early visual areas in the brain (Neggers et al., 2007; Van Ettinger-Veenstra et al., 2009; Gutteling et al., 2010, 2011, 2013). The close link between eye movement preparation and spatial visual attention depends on neuronal connections between the frontal eye fields and the visual cortex (Moore and Fallah, 2004; Ruff et al., 2006; Neggers et al., 2007; Van Ettinger-Veenstra et al., 2009; Gutteling et al., 2010; van Elk et al., 2010), supporting this hypothesis. Furthermore, an EEG study by van Elk et al. (2010) observed preparatory activity in occipital areas during grasping preparation, suggesting that a similar mechanism exists for manual motor actions. Using fMRI, increased activity was found in visuomotor areas of the parietal cortex when grasping was compared with simple reaching with the arm (Culham et al., 2003; Cavina-Pratesi et al., 2007), further confirming this notion.
A likely source of top-down feedback in case of grasping is the anterior intraparietal area (aIPS), which is involved in the planning and updating of grasping actions (Binkofski et al., 1999; Murata et al., 2000; Culham et al., 2003; Tunik et al., 2005; Baumann et al., 2009). Consistent with this, we recently demonstrated that the action-specific enhancement of orientation perception during grasping preparation is modulated when stimulating aIPS with TMS (Gutteling et al., 2013) just before action execution.
As orientation information is extracted in primary visual cortex (Hubel and Wiesel, 1974), we hypothesize that orientation processing in the early visual cortex is altered during grasping preparation, possibly as a consequence of feedback from aIPS, supporting subsequent visually guided action.
Here we aim to uncover influences of grasping preparation (the phase after a go cue, but before execution) in early visual areas using high resolution fMRI at 7T (tesla) and a surface-based multivoxel pattern analysis (MVPA) searchlight approach. Given that orientation perception is enhanced during grasping but not pointing preparation (Gutteling et al., 2011), we hypothesize that preparing to grasp an object enhances orientation sensitivity in the orientation representation of early visual cortex, whereas pointing at the object would not.
Materials and Methods
Subjects
Six healthy subjects (2 male, mean age 25.8, SD 2.5) with normal or corrected-to-normal vision participated in this study. Five of the six subjects were right handed. This study was approved by the Medical Ethical Committee of the University Medical Center Utrecht. All subjects signed an informed consent before participation.
Tasks
Subjects performed three tasks in two separate fMRI sessions. In one session, subjects performed a grasping and pointing task to oriented bars. In the other session, subjects engaged in a passive viewing orientation task, to isolate voxels selective for the orientations used in the grasping/pointing task, and a standard retinotopic mapping task to delineate early visual areas.
Grasping/pointing task.
Subjects were placed in the 7T MRI scanner, and a custom-made MR compatible “grasping/pointing device” shaped as a small table was placed over the lap. This device enabled subjects to perform grasping or pointing actions to two black bar objects mounted into a polystyrene foam extension with a vertical surface facing the subject, with white background. The two protruding bars were placed on either side of a fixation cross (6 cm from fixation cross to bar center, bar width 1.5 cm, length 6 cm), within easy reaching distance for the subject. The bars were oriented either at −45° (left) and +45° (right), or vice versa. This was counterbalanced across subjects. See Figure 1 for a graphical depiction of the setup. The device was visible through prism glasses, worn by the subject while lying supine. The same perspective was therefore obtained as when the subjects were looking down at their hands in a normal situation. Visual color cues were given by a back projection system, illuminating the white background on the device. Auditory cues (tones) were presented using speakers built into the MRI system through soft tubes inserted near the ear canal and closed off by moldable earplugs. The upper part of the right arm was restrained by a soft elastic band wrapped around the upper torso, to minimize head movement.
Paradigm. A, Graphic depiction of the grasping/pointing setup in the MRI scanner. Subjects lay supine in the MRI bore. A posterior volume was scanned (indicated in red), here depicted on a single subject anatomy (right). The partial volume was chosen to encompass both the early visual areas and the anterior parietal areas. Subjects were able to see the grasping/pointing display (bottom) through prism glasses. Color cues were given through a back projection system, illuminating the grasping/pointing display. When not giving cues, the back projection illuminated the grasping/pointing display (white). Every trial started with an instruction color cue (red/green, 1 s duration) indicating whether the action should be performed on the left or right bar. After a 2.5–3.5 s interval, an auditory cue instructed the subject to either perform the action (single beep) or withhold it (double beep). In case of a go cue, the subject performed the action that was instructed at the start of the block of trials. B, The retinotopic mapping stimuli consisted of a rotating checkerboard wedge (left) and an expanding checkered ring stimulus (middle). The black arrows did not appear in the actual stimulus presentation. The orientation grating stimuli (rightmost picture) consisted of two gratings of equal orientation, which appeared on either side of a fixation point for two seconds, followed by eight seconds of rest.
Hand movements were recorded with an MR compatible dataglove (Data Glove 5 Ultra MRI, 5DT) that tracked the bend fraction of the individual fingers through fiber optic tubes in the dataglove.
Subjects performed four grasping and four pointing blocks, each lasting 4 min, with a 20 s pause between blocks and a longer break halfway. Total duration was ∼35 min. Every block started with an instruction of the action to-be performed in the upcoming block (grasping or pointing) indicated by a color cue (red or green). Actions alternated for every block. Within each 4 min block, the subject performed 20 trials (a trial contains 1 grasp or pointing movement) of 12 s each. Each trial started with an instructional color cue, indicating whether subjects should perform their action toward the left of right bar object. After 3.5–4.5 s (random interval), an auditory cue instructed to perform the action (single beep, “go” cue). When a second beep sounded shortly afterward (400 ms) the movement that was prepared, had to be withheld (double beep, “no-go” cue). This resulted in six possible events per trial: grasping instruction, grasping go, grasping no-go, pointing instruction, pointing go, and pointing no-go. Subjects were asked to keep fixating a central cross throughout the experiment. When pointing, subjects were asked to point to the center point of the bar object. When instructed to grasp, thumb and index fingers were to be placed at the short sides of the bar. All visual and auditory stimuli were presented by the stimulus presentation software (Presentation v14.9, Neurobehavioral Systems), that also controlled the data glove recordings using a custom workspace extension.
Proper care was taken that grasping versus pointing classification results in the visual areas could not be explained by visual differences during grasping and pointing trials. The color cue that was used to instruct the target of action was the same for grasping and pointing. Left and right cues are averaged together during data analysis. The (no-)go cue was chosen to be an auditory signal only.
Orientation task.
In a separate session, subjects performed a passive viewing orientation task. Stimuli were presented using a back projection system and a Plexiglas screen mounted onto the receive coil, visible through a mirror system and prism glasses. Subjects viewed a fixation spot, flanked by two oriented gratings, which were both either 45° or −45°, see Figure 1B. The spatial frequency of the grating patterns was aimed to be approximately equal to the spatial frequency of the oriented bars in the grasping/pointing task. The task duration was 8 min and contained 48 repetitions (10 s each), equally divided between 45° and −45° gratings. At the start of each trial, the gratings were presented for 2 s, followed by 8 s of rest.
Retinotopy.
Subjects also engaged in a standard retinotopic mapping protocol (Engel et al., 1994; Sereno et al., 1995), consisting of a checkerboard stimulus and a rotating wedge (Fig. 1B). These tasks were done in a single session. Subjects viewed a rotating checkerboard wedge to obtain a polar angle map. The protocol started with 30 s of rest, followed by seven rotations of the wedge (thickness 48°), each 1 min long, and ended with another 30 s of rest. The eccentricity mapping was identical, with the exception that the main stimulus did not rotate, but expanded from the center outward once per cycle.
Acquisition
A high resolution 3-D segmented echo planar imaging sequence (voxel size: 1.25 mm isotropic; acquisition time: 2.88 s; 35 slices, echo time 25 ms) was used for all experiments. A posterior field-of-view was scanned, positioned such that the primary visual areas up to the anterior parietal regions were included (Fig. 1A). Therefore, the volume included the visual cortex and aIPS for all subjects. The volume was locally shimmed per subject to minimize geometric distortions in the functional volumes. All functional scans were acquired using a Philips 7T scanner (Philips Medical Systems) equipped with a 32-channel head coil. For the grasping/pointing task, 720 volumes were acquired and for the orientation task and each retinotopy task, 165 volumes. Whole brain T1-weighted anatomy scans were acquired on a Philips 3T Achieva scanner (Philips Medical Systems; voxel size 0.875 × 0.875 × 1 mm, 8 channel head coil). This anatomical scan was acquired at lower field due to intensity inhomogeneities at higher field, which interfered with the flat-mapping procedure. For registration purposes, a partial volume T1 was acquired for each subject at the 7T scanner (voxel size: 0.97 × 0.97 mm; slice thickness 1 mm, 44 slices), that encompassed the same volume as the functional images.
Analysis
GLM analysis was used to extract activation patterns from the grasping/pointing task, which were subjected to a surface-based searchlight MVPA analysis to find areas in the posterior brain that discriminate between grasping and pointing (preparation). Subsequently, a region-of-interest (ROI) analysis was performed to further examine the effects of action preparation on the early visual areas and aIPS, for orientation-selective patches. These orientation selective patches were obtained from the MVPA searchlight analysis of the orientation task. The ROIs were determined from the retinotopy and orientation task results.
Grasping/pointing task.
All acquired functional volumes were realigned using rigid body transformations. The partial T1 was coregistered with the functional scans using normalized mutual information. Subsequently, the whole brain T1-weighted anatomy obtained at the 3T scanner was coregistered to the partial T1 recorded at the 7T scanner. For MVPA analysis, multiple instances of an activation pattern were obtained from separate GLM analyses for all conditions of interest, i.e., the instruction, go and no-go conditions. The onset of the “instruction” regressor was at t = 0 (trial start), where a cue instructed the subject to perform an action to a left or right target. The go regressors were aligned to the first beep of the go cue, 3.5–4.5 s later. For no-go trials, the regressor was also aligned to the first beep, as this is where the action preparation starts, which we aim to capture. High-pass filter cutoff was 128 s. Each of these GLM analyses (e.g., the go-condition) contained a regressor for each of the conditions of no-interest (e.g., the instruction and no-go events) and multiple regressors for the condition of interest (e.g., the go grasping and pointing events). One regressor was included in the GLM for every four repetitions of one condition (e.g., four grasping-go trials) to reduce noise by averaging. These repetitions per regressor were spread out over the different blocks, to avoid biasing the pattern toward a certain block. Thus, one regressor would, for instance, consist of trial 1, 6, 11, and 16. With 160 repetitions, this resulted in 10 β images per action (grasping/pointing) for every condition (go/no-go). These 20 β images were used in the MVPA analysis to evaluate discrimination of grasping from pointing. Movement parameters and two white matter regressors (one for each hemisphere) were also added as nuisance regressors to the GLM to eliminate possible movement noise.
A surface based searchlight (Chen et al., 2011) MVPA was used. That is, a circular patch from which voxel values were sampled moved over the gray matter sheet, and these voxels were fed to the MVPA classifier for the various conditions (instruction, go and no-go). To accomplish this, two tissue segmentations of the whole brain anatomical 3T scan were made. First, the anatomical T1 weighted 3T scan was segmented using CARET software (Van Essen et al., 2001). This created a gray matter mask. Second, the same T1 scan was also segmented using unified segmentation (Ashburner and Friston, 2005) in SPM8 (http://www.fil.ion.ucl.ac.uk/spm/software/spm8/) to obtain a probabilistic gray matter map for each hemisphere. The CARET based segmentation was used as a mask for the probabilistic gray matter map, to restrict surfaces to a single hemisphere, remove the cerebellum, and remove any overlap between gyri. This combined probabilistic gray matter map was subsequently used to create a surface using SPM8 routines, expressed as a triangulation of the gray matter surface consisting of nodes (3D coordinates) and edges (connections, forming triangles).
For every node in the gray matter surface, a circular surface patch (6 mm radius) was calculated. The patches, calculated as a set of connected nodes, were interpolated to a set of voxels using a nearest neighbor algorithm. The surface based searchlight was iteratively moved over the entire gray matter surface coinciding with the imaged volume of the fMRI acquisition. The MVPA analysis was performed separately for every condition (instruction/go/no-go). The data instances (grasping and pointing activation patterns described above) were divided in a “training set” (18 patterns) and a “test set” (2 patterns). A linear support vector machine (LIBSVM implementation with a constant penalty parameter C = 1; Chang and Lin, 2011) was trained on the 18 training patterns (9 grasping, 9 pointing) and tested using the test set. Twentyfold cross-validation (Kohavi, 1995) was performed to assess classifier performance. The resulting accuracy was attributed to the center voxel for each patch. This procedure was repeated for every voxel in the scanned volume to obtain a classification accuracy map for all conditions (instruction/go/no-go) and all subjects.
Orientation task.
The surface based searchlight procedure was also applied to the functional volumes obtained from the orientation task. Here, β images were obtained from a GLM analysis, where regressors for −45° and 45° presentation were used (aligned at the onset of presentation, duration 2 s). As in the grasping/pointing task, these were also estimated based on four repetitions, but as there were 48 repetitions in total, this resulted in a total of 12 β images (6 for −45°, 6 for 45°). Surface based searchlight classification was performed on the categories −45°/45°.
Retinotopy.
Processing of both eccentricity and polar angle functional data were performed in the same way. The volumes were realigned and coregistered with the full T1 anatomical scans (acquired at 3T) using the partial T1 from the 7T as an intermediate. The data were smoothed with a 4 mm FWHM Gaussian kernel. In SPM, the data were fitted to a sine and cosine regressor. For every pair of β voxel values (corresponding to the sine and cosine regressor), the arctangent was calculated to obtain an eccentricity/polar angle phase map (Engel et al., 1994; Sereno et al., 1995).
These polar angle and eccentricity maps were projected to the CARET generated surface representation and warped to a flat map. Using the polar angle phase reversals and direction of eccentricity, ROIs were manually created for early visual areas V1–V3. Furthermore, an ROI of the aIPS was created based on cortical landmarks (the confluence of the anterior end of the intraparietal sulcus and the postcentral gyrus; Culham et al., 2003; Cavina-Pratesi et al., 2007).
ROI analysis.
The ROIs described above were used to analyze the grasping/pointing task. For every ROI, only those voxels were selected that exhibited significant above chance classification performance in the orientation task (and can hence discriminate between ±45°). To this end, first the classification accuracy level to represent voxels “significantly above chance” was established using permutation testing (Golland and Fischl, 2003; Bonferroni corrected α threshold of 0.05). Second, a mask was created for each ROI that only contained those voxels that were significantly above chance for orientation classification. Third, these masks were used to extract the surface based searchlight accuracy values from the grasping/pointing task. That is, only accuracies from searchlight center-voxels from the grasping/pointing task were selected that were significant in the orientation coding task. These grasping versus pointing discrimination values were averaged, to obtain the mean accuracy for each ROI, condition and subject separately.
Laterality analysis.
To check whether the effects of action preparation are lateralized, i.e., whether actions performed toward a target on the right show a different lateralization pattern than actions performed to the left, the ROI analysis was repeated for each action direction. The analysis performed was identical to the standard ROI analysis, except that analyses were performed with only one-half the data for each action direction (left/right).
Results
Decoding of action intention
We obtained up to 80% classification scores for grasping versus pointing actions in the superior parietal areas, extending into the intraparietal sulcus and in the occipital lobe, along the calcarine sulcus, lateral occipital areas, and the parieto-occipital junction. Figure 2 provides an overview of the results for grasping and pointing (go/no-go). In general, high grasping versus pointing decoding performance was found throughout the parietal and occipital areas for both go and no-go conditions. These regions thus encode different information for grasping and pointing movements during action preparation, whether the action is executed or not. Figure 3 shows that timing in the go condition was very similar for grasping and pointing, and that subjects were well able to abort execution of the action when they perceived a no-go cue. The parietal areas that decode grasping versus pointing correspond well with established areas of the parietal reaching and grasping network (Culham and Valyear, 2006; Andersen and Cui, 2009). Importantly, high grasping versus pointing decoding performance was also found in early occipital areas, demonstrating that action preparation affects early visual processing, as hypothesized.
Grasping/pointing Searchlight results. Group results for the surface based searchlight grasping versus pointing analysis on a normalized brain. A, Classification results for the cue go condition, where action was executed. B, Classification results for the cue no-go condition, where the action had to be withheld. C, Classification scores during the instruction phase. The individual searchlight results were normalized, averaged across subjects, and smoothed for display purposes. The left hemisphere is depicted on the left (lateral and medial view) and the right hemisphere on the right. The color map represents classification scores.
Kinematics. Grand averages of the obtained hand movements during the grasping/pointing task, recorded using a dataglove during the experiment. The left graph shows finger bend fractions during grasping for the index finger and thumb, in both go (solid line) and no-go (dotted line) conditions. Time 0 represents the auditory go-cue. In the no-go condition, an auditory stop-cue followed shortly after the first. The right graph depicts the movements during the pointing movement for the index finger in the go (solid line) and no-go conditions (dotted line).
Decoding of grating orientation
The orientation decoding experiment was intended to isolate voxels that can classify rightward tilted gratings from leftward tilted gratings. Here, classification performance was generally lower than for grasping versus pointing classification. Orientation decoding results from a single subject are shown in Figure 4. All subjects showed similar decoding performance, although there is considerable variation in locations that yielded good classification performance between subjects, yielding little overlap in the mean surface map. The significance threshold for orientation classification was 72.2% accuracy and varied only marginally between subjects (SD 0.4%). As well as sites in early visual cortex, high orientation decoding performance was also seen in the superior parietal and intraparietal areas. Only voxels that can classify rightward from leftward grating orientation were used for subsequent ROI analysis of data from the grasping/pointing run for each subject separately, to ensure we are looking at units that process orientation information.
Orientation decoding results. Single-subject results of the surface based searchlight analysis of the orientation task in a medial view and flat map. A classifier was trained to discern gratings of 45° or −45° (leftward vs rightward tilt) for every surface based searchlight patch. The dotted red line indicates the edge of the scanned volume. Individual regions of interest of early visual areas V1–V3 are marked. The color map represents classification scores.
ROI analysis of V1–V3 and aIPS
A ROI analysis was performed to further examine the effects of action preparation on the early visual areas and aIPS, for orientation-selective patches. V1, V2, and V3 ROIs for each subject were created using conventional retinotopic mapping procedures (Engel et al., 1994; Sereno et al., 1995). Within these ROIs, searchlight classification accuracy values of grasping versus pointing were extracted for voxels that showed significantly above chance orientation classification performance, and those values were averaged across voxels within the ROIs. Results averaged over subjects are shown in Figure 5. Classification performance for discriminating grasping from pointing actions is significantly above chance (50%) for all regions of interest [Bonferroni corrected α of 0.0021, one sample t tests (24), all p < 0.00071].
ROI analysis. Classification scores of the grasping/pointing task for every ROI, thresholded to include only surface patches that can significantly classify orientation. For both hemispheres, mean classification values are depicted for the go condition (where a grasping or pointing action was performed), no-go condition (where the action was withheld), and instruction condition (the cue instructed the action target). Error bars denote the SEM. All classification scores are significantly above chance level (50%) corrected for multiple comparisons (Bonferroni corrected α of 0.0021, one sample t tests, all p < 0.00071).
When an actual movement was executed (the go condition), classification scores are higher for aIPS than the early visual areas. The early visual areas V1–V3 show high classification accuracy bilaterally.
When the action was prepared but not executed (no-go condition), classification scores are generally lower than for actual actions (go condition), especially in the aIPS. Interestingly, when looking at grasping versus pointing classification performance during the instruction phase, high classification scores are also observed for V1, V2, and V3. Instruction events indicate the grasping/pointing target location, ∼3 s before the go or no-go cue. However, in aIPS the grasping versus pointing classification performance is lower at the time of instruction than during the go or no-go cue. Classification scores in the aIPS differ significantly between the go and instruction phase (one-tailed paired sample t tests; left hemisphere: p = 0.048, right hemisphere: 0.043), whereas there is no significant difference between these phases in V1 (left hemisphere: p = 0.153, right hemisphere: p = 0.182). This suggests that aIPS activity reflects information more closely related to the actual motor control signals, whereas the early visual areas' activity reflects visual information processing required for the upcoming act. This is in agreement with the general notion that aIPS is at the interface of visual perception and action (Culham et al., 2003; Cavina-Pratesi et al., 2007).
Laterality analysis
We repeated the ROI analysis for each action direction, i.e., for targets presented on the left and right, see Figure 6. Note that all actions are still performed with the right hand. Although it would be expected that the effect of action preparation would be strongest in the contralateral visual area, suggesting a retinotopic organization, no such consistent pattern was found. Classification scores in the no-go condition are notably lower for targets on the left, but the opposite pattern is not present for targets on the right. As would be expected, high decoding performance is found in the contralateral aIPS, for targets on the right.
Laterality analysis. Classification scores of the grasping/pointing task for every ROI, as in Figure 5, but split between actions performed to targets to the left (top) and right (bottom). The ROIs are denoted by their hemisphere (e.g., “Left aIPS” refers to the aIPS in the left hemisphere). Error bars denote the SEM.
Discussion
We were able to predict whether a subject was going to grasp or point to a bar object from fMRI signal modulations in early visual cortex, including V1, using MVPA. These results support the hypothesis that action preparation modulates visual processing of action relevant features as early as V1.
As an additional constraint, only voxels that responded significantly to the orientations of grasping targets (tested using Gabor patches of 45° and −45°) were included in the analysis. As orientation perception improves when preparing a grasping action (Craighero et al., 1999; Gutteling et al., 2011), we expected that groups of voxels that are highly sensitive to the orientations used in this study would show the hypothesized action preparation effects. Our results show that all regions included in the ROI analysis yielded above chance classification scores, during the preparation of an action and even during target instruction several seconds before that. This supports the notion that changes in activity in these early visual areas underlie action induced perceptual modulations. Unfortunately, our design precluded testing whether the bar orientations could be classified more accurately when grasping than when pointing (i.e., classify −45° vs 45° bars), as would be expected. The orientation of the bars were confounded with the direction of the action (left/right), hence further studies are needed to test this hypothesis.
Classification scores are highest in the cue go condition, where classification is performed on data acquired during the preparation phase of actual movements. This is the most engaging condition requiring maximal visuomotor processing. Here, the aIPS shows the highest classification scores. Furthermore, aIPS shows a specific pattern of classification scores across different conditions (go/no-go/instruction), which is different from the pattern across conditions shown by the early visual area V1. This indicates that the grasping/pointing signals classified in the aIPS are directly related to action execution. Note, however, that the hand was visible during action execution. Although the difference in visual input was minimal, this may also be a source of information used by the classifier.
In the no-go condition the action was prepared but not executed. This enables analysis of neuronal processes related to action preparation, while excluding interpretations where activity differences result from action performance or vision. Note that here we refer to “action preparation” as the period just after a go-cue, where the action is prepared, but not yet executed, as we found this to be a critical time period for the proposed feedback mechanism (Gutteling et al., 2011, 2013). Even though the action is cancelled by the second (no-go) cue, action preparation is present in this condition. For the no-go condition, classification scores were also clearly above chance level. Although the classification scores are generally lower for no-go trials, in occipital ROIs they are very similar to the go condition, demonstrating that actual action performance has little effect on early visual cortex. However, the classification score in the aIPS was significantly lower when no action was actually executed. This is consistent with the hypothesis that aIPS activity depends more on motor execution processes, similar to areas in the dorsal processing stream, which are known to show both visual and direct action control signals in monkeys (Murata et al., 2000). The grasping versus pointing classification performance observed in aIPS could therefore be related to the hypothesized feedback signal to the visual areas or to direct action control. The observation that the grasping and pointing actions can still be robustly classified in the absence of the actual action indicates that action preparation itself is sufficient to enhance processing of action-relevant information in occipital visual areas. Previous findings have shown connections from frontoparietal areas providing feedback to early visual areas (Moore and Fallah, 2004; Ruff et al., 2006, 2008), revealing a topographical organization. In the current study, we did not observe a consistent lateralization pattern when separating targets on the left and right. This may be due to the increased noise when using half of the dataset. Alternatively, the retinotopic organization of feedback for eye movements (e.g., originating from the frontal eye fields) may be stronger than for hand actions (e.g., originating from aIPS), as shown by Ruff et al. (2008).
The classification scores in aIPS are notably lower during the instruction phase than immediately following the go or no-go cues occurring later. Although in this case there may not be a direct driving input modulating the activity in V1, there may be a baseline shift in visual processing favoring processing of grasping-relevant orientations at the expense of other information. Grasping and pointing were performed in separate blocks of trials lasting 4 min in total, instead of an intermixed design. Without the need to switch, subjects may have been in a “grasping-mode”; i.e., a state of awareness related to having to grasp an object. This would entail a steady-state of improved perceptual sensitivity for grasping-relevant features, allowing for classification of action intention, several seconds before the go (or no-go) cue. Such “action modes” for our perceptual system might be a neuronal implementation of persistent biases in perception that seem to exist for persons skilled in a certain action (Witt, 2011). Alternatively, subjects may have engaged in motor imagery during the instruction phase. Although the early visual areas studies here are not typically associated with motor imagery activity (Mizuguchi et al., 2013, 2014), the (preparatory) activation of more motor-related areas may have provided early feedback to the visual areas regarding the action relevant features, even before the go-signal.
The surface based searchlight analysis of the entire scanned volume revealed that large parts of the superior parietal and intraparietal areas yielded grasping versus pointing classification scores equal or higher than those obtained for the visual areas. These areas are involved in the dorsal stream of the frontoparietal reaching and grasping network (Culham et al., 2003; Fogassi and Luppino, 2005; Tunik et al., 2008; Cohen et al., 2009). As aIPS is our main parietal area of interest because of its implication in grasping preparation (Culham et al., 2003; Cavina-Pratesi et al., 2007; Gutteling et al., 2013), we included it in the ROI analysis. However, any area that has a different activity pattern for grasping actions than for pointing actions will likely show above chance classification. Thus, areas (specifically) involved in pointing movements, such as the “parietal reach region” (Andersen and Buneo, 2002; Calton et al., 2002), V6A (Galletti et al., 2003), and the superior parieto-occipital complex (Gallivan et al., 2011) will also show high classification accuracy. AIPS showed high classification scores, supporting a possible role in generating modulatory feedback signals to early visual cortex. Recent studies have shown projections from parietal areas, including IPS, to early visual areas (Borra and Rockland, 2011; Greenberg et al., 2012). This notion is further supported by our recent study where aIPS was stimulated with TMS (Gutteling et al., 2013), effectively abolishing enhanced orientation perception during grasping preparation as we observed in earlier work (Gutteling et al., 2011).
Importantly, pattern analysis techniques are very sensitive to any kind of consistent difference in the fMRI signals between movement conditions, either of neuronal or non-neuronal origin. First, the direct effect of movement or body position in the magnetic field of the scanner, causing inhomogeneities of the B0-field, can affect the signal in the acquired volume. In the go condition, the actual movement following shortly after the go cue was unlikely to co-occur with the BOLD response of interest that is delayed for several seconds (Logothetis et al., 2001) and subjects returned to the start position immediately after action performance. Furthermore, in the no-go condition, the movement was near fully suppressed as evidenced by the dataglove recordings, but still clear grasping versus pointing decoding performance was observed in all ROIs. Second, care was taken during the acquisition and analysis to eliminate head movements that often co-occur with arm movement. The upper arm was strapped to the torso, to avoid movement being carried over to the head. In the analysis, both realignment parameters and two white matter regressors (one for each hemisphere) were added as nuisance regressors, effectively removing head-movement biases. We also ensured that there was no visual difference between the stimuli used in grasping and pointing trials, which might otherwise have explained the results. Unfortunately, eye tracking could not be performed during the experiment, as the prism glasses precluded this. Although subjects were instructed to maintain fixation at all times, eye movements may be present during action execution. However, it is unlikely that this is a significant source of classification information in the no-go and instruction conditions, where only fixation was required.
The phenomenon of action-preparation induced visual processing enhancements should not be considered in isolation. In general, the extensive body of literature on selective visual attention (Posner, 1980) indicates that our visual field is not processed equally for all locations and features. Rather, by means of visual saliency (bottom-up attention) or task requirements (top-down attention), different parts of the visual information will be selectively processed (Treisman and Gelade, 1980). Also, a link between attention and action preparation has been suggested in influential theories, such as the premotor theory of attention (Rizzolatti et al., 1987) and selection-for-action (Allport, 1987). Furthermore, neuronal signals in early visual areas can be modulated by top-down task demands originating elsewhere in the brain (McAdams and Maunsell, 2000). Interestingly, recent studies that investigated action planning using multivariate techniques have uncovered extensive action planning activity, not only in dorsal stream areas (Gallivan et al., 2013b) but also in ventral steam areas (Gallivan et al., 2013a) not previously found using univariate techniques. This finding fits well with the idea of a neuronal mechanism proposed here, which reflects a novel and specialized form of top-down signal related to the preparation of an imminent grasping action, specifically and functionally aimed at perceiving object orientation optimally to facilitate the upcoming grasp. Whether feedback reaches the ventral stream through direct connections or through the early visual areas remains unclear.
In conjunction with other recent reports (van Elk et al., 2010; Gutteling et al., 2013), a coherent picture emerges that several specific neuronal top-down signals related to different forms of motor control exist. Such action preparation signals modulate visual processing and perception according to that action's needs. These signals are most likely originating from activation in higher-order visuomotor control brain areas downstream from early visual cortex in the dorsal processing stream.
In summary, we conclude that the high grasping versus pointing classification scores in V1 and the other early visual areas V2 and V3 are due to modulatory influences related to action preparation. This modulation may be instantiated by a shift in orientation tuning for neurons in the orientation columns (Kamitani and Tong, 2005; Norman et al., 2006; Jehee et al., 2011), or on a larger cortical scale by changes in radial bias of orientation representation (Sasaki et al., 2006; Freeman et al., 2011). Both explanations, however, imply neural changes in the orientation representation in early visual cortex due to action preparation. The current results show strong evidence that grasping preparation modulates orientation selectivity in the early visual areas, enabling action-modulated perception effects that improve processing of action relevant visual features.
Footnotes
This work was supported by a Netherlands Foundation for Scientific Research (NOW; http://www.nwo.nl/) Open Competition Grant NWO 400-05-134.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr Tjerk Gutteling, Radboud University Nijmegen, Donders Institute for Brain, Cognition and Behaviour, Montessorilaan 3, 6525 HR Nijmegen, the Netherlands. t.gutteling{at}donders.ru.nl
