Abstract
In this time-resolved functional magnetic resonance imaging (fMRI) study, we aimed to trace the neuronal correlates of covert planning processes that precede visually guided motor behavior. Specifically, we asked whether human posterior parietal cortex has prospective planning activity that can be distinguished from activity related to retrospective visual memory and attention. Although various electrophysiological studies in monkeys have demonstrated such motor planning at the level of parietal neurons, comparatively little support is provided by recent human imaging experiments. Rather, a majority of experiments highlights a role of human posterior parietal cortex in visual working memory and attention. We thus sought to establish a clear separation of visual memory and attention from processes related to the planning of goal-directed motor behaviors. To this end, we compared delayed-response tasks with identical mnemonic and attentional demands but varying degrees of motor planning. Subjects memorized multiple target locations, and in a random subset of trials targets additionally instructed (1) desired goals or (2) undesired goals for upcoming finger reaches. Compared with the memory/attention-only conditions, both latter situations led to a specific increase of preparatory fMRI activity in posterior parietal and dorsal premotor cortex. Thus, posterior parietal cortex has prospective plans for upcoming behaviors while considering both types of targets relevant for action: those to be acquired and those to be avoided.
Introduction
Prospective plans of future action are initially represented in an extrinsic (visual) rather than an intrinsic (effector-specific) reference frame (Morasso, 1981; Wolpert et al., 1995), making it difficult to extract these earliest precursors of behavior from the neuronal representation of the sensory context in which they are embedded. To make such a distinction, experimenters have used so-called precuing tasks with delayed behavioral responses (Rosenbaum, 1980). These delayed-response tasks allow the separation of planning processes from both (1) the sensory representations on which they depend and (2) the motor acts they produce. This separation is achieved by briefly providing the sensory “context cue(s),” relevant for the planning of an upcoming behavior, while interdicting the actual execution of this behavior until the later presentation of a “go signal,” thereby temporally isolating planning processes within the intervening “delay” or “memory” period. Thus, sustained neuronal activity recorded during this delay period would refer to neither the immediate processing of the sensory context nor the actual movement performance (Hikosaka and Wurtz, 1983). Rather, sustained activity would reflect isolated processes related to the planning of an upcoming movement (Connolly et al., 2002; Thoenissen et al., 2002; Astafiev et al., 2003; Brown et al., 2004; Medendorp et al., 2005, 2006; Schluppeck et al., 2006; Connolly et al., 2007). Among the areas that were reported to exhibit sustained activity throughout the preparatory delay period, human posterior parietal cortex (PPC) plays a dominant role. Situated between visual and somatosensory cortex and with connections to premotor cortex and the aforementioned sensory areas, PPC has the anatomical prerequisites that qualify it as a candidate structure for visual action planning. Moreover, both the properties of neurons within PPC (Andersen and Buneo, 2002) as well as the symptoms of patients with lesions of this part of the brain (Perenin and Vighetto, 1988; Rushworth et al., 2003; Karnath and Perenin, 2005; Trillenberg et al., 2007) further support this notion. However, the idea that PPC, as a part of the dorsal visual pathway, would primarily contribute to behavior (Goodale and Milner, 1992) is in sharp contrast with an alternative interpretation of dorsal stream function (Mishkin and Ungerleider, 1982), which rather highlights the processing of visuospatial sensory information for sensation. Following these dichotomous views, the question arises whether sustained PPC activity in delayed-response tasks is (1) prospective or (2) retrospective in nature: does PPC organize a future behavior in a prospective manner or, alternatively, does PPC process and maintain retrospective sensory information as suggested by various recent investigations (Rowe et al., 2000; Simon et al., 2002; Curtis et al., 2004; Volle et al., 2005; Curtis and D'Esposito, 2006; Mars et al., 2008)? To answer this question, we devised novel variants of the delayed-response paradigm while monitoring brain activity using event-related functional magnetic resonance imaging (fMRI).
Materials and Methods
Subjects.
Eight subjects (five males, three females) participated in the experiment. All had normal or corrected-to-normal visual acuity, and all except one subject were right-handed. The latter subject performed equally well with both her left and her right hand. Participants provided informed consent in accordance with the declaration of Helsinki and the California Institute of Technology Institutional Review Board guidelines. Subjects were reimbursed for participating in this experiment and received $10/h.
Task design.
Subjects were scanned in three consecutive runs while performing variants of a delayed-response task. In all task variants, subjects were instructed to perform sequences of “virtual reaches” with a computer cursor. The cursor was controlled by an fMRI-compatible trackball that was operated with the subjects' right index finger. In one of the randomly interleaved tasks, subjects were only required to memorize either two or four target locations. In other tasks, the same targets either instructed the goals for an upcoming movement sequence or, alternatively, they instructed movements that should be avoided. Hence, by varying the prospective planning demands across tasks while keeping the mnemonic requirements constant, we aimed to isolate delay-related fMRI activity reflecting the preparation of an upcoming behavior.
Specifically, to engage movement planning in a subset of trials, subjects performed a “classic” version of the “delayed-response task” (DRT). In this task, movements were instructed by either two or four cues that were presented before the delay period (Fig. 1A). After the delay, subjects had to move a trackball-controlled cursor as fast as possible to each of the remembered cue locations to successfully complete the trial. Importantly, in this (and each other) experimental condition, response times were limited to further encourage preplanning of the required behavioral response. Furthermore, central fixation was instructed throughout all trials.
The behavioral tasks. Each of the randomly interleaved tasks started with a fixation period of random duration that served as a baseline epoch for our fMRI analysis. In the following cue period, varying numbers of peripheral targets (open squares) were presented. A–D, Depending on the color of the central fixation spot, these targets instructed goals for an upcoming movement (A), signified undesired locations (B), marked spatial positions that had to be memorized (C), or served as irrelevant distractors (D). The cue period was followed by a random-dot pattern (mask) that masked putative after-images of the cue. During the delay period, subjects were asked to prepare for the upcoming response. A, In the DRT, movements had to be planned to all remembered target locations. B, In the NM2ST, subjects avoided movements to all precued locations since they had to perform reaches toward all those targets in the response period that had not been shown before. C, In the M2ST, subjects just needed to memorize the initial cues to perform a delayed match-to-sample comparison (match). In this task, subjects could not prepare or inhibit specific movements during the delay period because the response was ultimately determined by a randomly generated response screen that appeared at the end of the trial: subjects were asked to indicate a match/mismatch by reaching toward the white/black targets, respectively. D, In the CT, which controlled for nonspecific preparatory activity during the delay, the initial cues were irrelevant and could be ignored. Here the subjects' task was to reach toward all targets that were shown during the response phase. For all conditions, the durations of the respective task epochs are indicated above the sketches, and values separated by slashes refer to two-target/four-target conditions, respectively. Note that we increased the relative size of selected stimulus elements depicted for illustrational purposes.
In a second task, the “non-match-to-sample task” (NM2ST), sample cues signified undesired target locations, and the required motor response was only later defined by a second set of randomly selected cues that were presented immediately after the delay period. Subjects were instructed to perform movements toward all the new targets within the second set, i.e., those targets that were not already present in the first set of sample cues (Fig. 1B). Because this procedure ensured that subjects were unable to predict the required sequence of finger reaches during the preparatory delay, several previous studies have referred to comparable tasks as controls for retrospective visual memory. However, one cannot rule out the possibility that subjects prepare potential actions to the remaining target locations or inhibit actions to the precued, undesired target locations (or both). Accordingly, we considered the NM2ST a putative planning condition and probed for potential prospective processes during the delay period.
To control for retrospective visual spatial memory of the target cues as well as covert shifts of attention toward these cues, we devised a “match-to-sample task” (M2ST). In this task, the initial cues neither predicted the required motor responses nor did they limit the number of response alternatives. Instead, subjects simply needed to memorize the cue locations to compare them with a second set of cues that was presented after the delay. If both sets were identical (50% of trials), subjects had to move the cursor toward the white targets of a randomly generated third set of response cues. If the two sets differed (in 50% of trials, the location of one target was altered), subjects were instructed to move the cursor toward the black targets of the response set (Fig. 1C). Hence, any retrospective processes (or “retrospective task demands”) in the M2ST were identical to those in the DRT and in the NM2ST. Thus, any brain area that (compared with the M2ST) would exhibit increased levels of delay-related fMRI activity in the DRT, the NM2ST, or both tasks should participate in the prospective organization of the required behavioral responses.
Finally, the “control task” (CT) served as a “baseline condition.” In this task, the initial cues (all nine cues) were irrelevant for the subsequent motor response. Subjects just had to move the cursor toward the targets that were presented immediately after the delay period (Fig. 1D). This task controlled for visual responses and nonspecific motor preparation common to all tasks.
Stimulus presentation.
All visual stimuli were backprojected onto a translucent screen (22° × 16° visual angle) using a video projector (800 × 600 pixels, 60 Hz). Subjects viewed the visual stimuli via a mirror that was mounted on the head coil of the MRI scanner (viewing distance of 1150 mm). Stimuli were generated on a windows personal computer using “Cogent Graphics” developed by John Romaya at the Laboratory of Neurobiology at the Wellcome Department of Imaging Neuroscience (London, UK).
Each trial started with a random fixation period (14,000, 15,000, or 16,000 ms) during which a white fixation cross in front of a background square (0.8° × 0.8°) was presented on the otherwise dark screen. This fixation cross always remained visible throughout the whole trial. The initial fixation period was followed by cue presentation (500 or 1500 ms). The length of cue presentation was determined by the number of targets being presented: it was longer for four targets compared with the two target trials to provide sufficient time for target encoding and thus to guarantee comparable performance rates. All potential target locations were shown as equally spaced gray squares (0.8° × 0.8°, angular distance of 40°) that were arranged on a circle around the fixation spot (5.5° radius). Across trials, this overall circular arrangement was randomly rotated by either −10° or 10°. Both this rotation as well as the asymmetric arrangement of nine targets should prevent subjects from forming “clock-related” or other verbal memory strategies. Actual targets were defined as open squares (inner black square, 0.3° × 0.3°). Cue presentation was followed by a 1000 ms mask to prevent after-images of the cues. This mask consisted of 80 randomly placed white squares that densely covered the relevant central part of the screen (14° × 14°). Afterward, there was a delay period of random duration (14,000, 15,000, or 16,000 ms) in which subjects should maintain central fixation on the otherwise dark screen while preparing for the upcoming response. The long durations of the delay periods were required to allow a time-resolved analysis of preparatory fMRI activity without being confounded by preceding visual cues.
In all conditions, except the M2ST, the delay period was immediately followed by the response period. In the response period, subjects had to perform a speeded sequence of out-and-back reaches with a circular cursor (1.65° diameter). Cursor movement was controlled by an MRI-compatible trackball (Current Designs) that was placed at a comfortable distance on the subjects' belly. Subjects operated the trackball with their right index finger and were not allowed to lift their finger from the trackball during a trial. The sensitivity of the trackball interface was adjusted in a way that subjects could easily perform out-and-back “cursor saccades” through straight, quasi-ballistic index finger movements, without a need for readjustment of the trackball between individual movements (supplemental Figs. S1, S2, available at www.jneurosci.org as supplemental material). Subjects' finger movements were continuously monitored throughout the trial with a sampling frequency of 60 Hz while the cursor was only visible during the response phase, serving as a start signal for the response. The (invisible) starting position of the cursor was always re-centered at the beginning of each trial. During the response period, subjects were free to choose any of the relevant targets for their initial reach and then had to complete the sequence by reaching toward each remaining target in a continuous manner, i.e., in either a clockwise or a counterclockwise order (Fig. 1). We prompted subjects to perform out-and-back movements to minimize the need for spatial updating. Moreover, requiring multiple out-and-back finger reaches should guarantee highly demanding and long-lasting movement preparation during the rather lengthy delay epochs. Subjects were instructed to perform the sequence of movements as fast as possible, since the time for completing the instructed response was highly limited. The time remaining for the manual response was indicated by the cursor itself, which gradually faded until it became invisible at the end of the response period. The length of this period was individually adjusted for each condition: it was longer the more movements were required and the more cognitive processes had to be performed. The respective times were chosen based on average reaction times and movement time measures obtained in a pilot study (n = 10, including 6 subjects of the current study; note that the same pilot study was also used to adjust cue duration in a way that performance levels between two-target and four-target conditions were approximately matched).
In the M2ST, the response period was preceded by a “match” stimulus that had to be compared with the earlier presented cues, serving as the sample. Moreover, during the response period of this task, two different sets of randomly selected targets served to indicate the outcome of the match-to-sample comparison: movements toward white targets would indicate a match, whereas movements to black targets would signify a mismatch. The response period of this and all other tasks was followed by a 2000 ms intertrial interval and another 1000 ms random-dot mask. During the intertrial interval and during mask presentation, subjects were allowed to blink or to look anywhere on the otherwise dark screen. All trial types were presented in a pseudorandomized order. Each subject performed an overall of 18 trials per condition that were acquired in three experimental runs; in half of the trials, we showed only two targets, whereas four targets were presented in the other half (i.e., nine trials each). All subjects performed one training run (six trials per condition) inside the scanner before functional imaging. Detailed feedback about behavioral performance was provided after training and after each experimental run.
Performance monitoring and analysis.
Our experimental paradigms required subjects to perform finger reaches while maintaining central fixation. Eye movement recordings were made with an fMRI-compatible infrared eye camera (Resonance Technology) and ViewPoint Eye Tracker software (Arrington Research). Eye position was sampled at a frequency of 60 Hz. Additional processing of the behavior was performed offline using Matlab 7.5 (MathWorks). Eye position samples were filtered using a second-order 10 Hz digital low-pass filter (Chebyshev type II, r = 3). Saccades were detected using an absolute velocity threshold (20°/s), whereas blinks were identified as gaps in the eye position records caused by lid closure.
As with eye movements, cursor/trackball movement was expressed in degrees of visual angle to allow direct comparison. Representative examples of two-dimensional cursor and eye movement recordings are provided in supplemental Figure S1A (available at www.jneurosci.org as supplemental material). Cursor movements were low-pass filtered at 24 Hz. For reaction time estimates, we calculated the absolute cursor velocity. Reaction time was defined by the time that elapsed after the appearance of the visual movement cursor and until response onset, i.e., the time when absolute cursor velocity exceeded a threshold of 8.1°/s (supplemental Fig. S1C, available at www.jneurosci.org as supplemental material). Importantly, in none of the subjects were suprathreshold movements detected during the delay phase (compare with supplemental Fig. S1B, available at www.jneurosci.org as supplemental material). In addition, there was no evidence for differences in subthreshold absolute trackball velocity across conditions (compare with supplemental Fig. S2, available at www.jneurosci.org as supplemental material).
Finally, performance was expressed as error rates: if the average direction of any individual out-and-back reach within the sequence would not be located within the respective 40° angular target sector, this would result in an error trial. However, because there were no systematic differences in error rates among the conditions of interest (DRT, NM2ST, and M2ST; see Results) and because even in error trials all subjects performed 50 or 75% of the reaches into the correct target window, we did not treat these trials differently in our fMRI analysis.
To account for our within-subject design, behavioral performance was analyzed using two-way repeated measures ANOVA (rmANOVA2) with factors condition (DRT, NM2ST, M2ST, and CT) and target load (two and four targets). We tested for sphericity (Mauchly's test) and adjusted the F statistic according to the procedure of Greenhouse and Geyser whenever the assumption of sphericity was met. If the factor condition was significant, we performed additional post hoc comparisons using paired t tests that were Bonferroni's corrected for multiple comparisons. All post hoc comparisons that turned out significant are indicated in the respective figures and the supplemental data (available at www.jneurosci.org as supplemental material).
In all related figures, we provide averages, calculated across subjects' normalized measures: as a result of our within-subject design, normalization was performed to eliminate the between-subjects variance. Toward this end, we performed a procedure that was suggested by Masson (2003): “Normalization is based on the deviation between a subject's [i] overall mean [mi], computed across that subject's scores in each condition, and the grand mean [GM] for the entire sample of subjects […]. That deviation is subtracted from the subject's score [X] in each condition [j] (i.e., Xij − (Mi − GM)) to yield a normalized score for that subject in each condition […].” Note that this normalization does not change across-subject averages. However, the measures of variability depicted in our figures, here the SEM, exclude the between-subject variance and refer to the average, within-subject variability only.
Image acquisition and fMRI analysis.
MRI images were acquired on a 3 T Siemens TRIO scanner using an eight-channel head coil (Siemens). For each subject, we obtained a T1-weighted magnetization-prepared rapid-acquisition gradient echo (or MP-RAGE) anatomical scan of the whole brain (176 slices; slice thickness, 1 mm; gap, 0 mm; in-plane voxel size, 1 × 1 mm; repetition time, 1500 ms; echo time, 3.05 ms; field of view, 256 × 256 mm; resolution, 256 × 256 voxels) as well as T2*-weighted gradient-echo planar imaging scans [echo planar images (EPIs): slice thickness, 3.5 mm; gap, 0 mm; in-plane voxel size, 3 × 3 mm; repetition time, 2000 ms; echo time, 30 ms; flip angle = 90°; field of view, 192 × 192 mm; resolution, 64 × 64 voxels; 32 axial slices) for our fMRI time-series analysis. Overall, we obtained 1458 EPIs per subject, which were collected during the three consecutive runs of ∼16 min length each. The EPI volume completely covered the cerebral cortex as well as most subcortical structures. Only the more posterior aspects of the cerebellum were dropped in several of our subjects.
Functional image processing was performed using SPM2 (Wellcome Department of Cognitive Neurology, London, UK). Images of each subject were realigned by using the first scan as a reference. T1 anatomical images were coregistered to the mean image of the functional scans and then aligned to the SPM T1 template in MNI space (Montreal Neurological Institute, mean brain). The resulting nonlinear three-dimensional transformation was applied to all images for spatial normalization. Finally, the functional images were spatially smoothed with a Gaussian filter (7 × 7 × 7 mm3 full-width at half-maximum) and high-pass filtered (cutoff period, 128 ms). Functional images were analyzed on both the individual subject and group levels. In the subject-specific analysis (first level), we specified a general linear model (GLM) including regressors for each of our eight different tasks (4 conditions × 2 target loads) and for each task epoch (combined cue and mask presentation, delay period, response period). All regressors were convolved with the default canonical hemodynamic response function. The intertrial interval and the initial fixation period were not explicitly modeled and served as the baseline epoch. For each subject, we calculated statistical contrast images for the following effects: (1) delay activity, [DRT, M2ST, NM2ST] > [CT]; (2) inverse control, [CT] > [DRT, M2ST, NM2ST] (this contrast did not reveal any significant activation); (3) movement planning, conjunction {[DRT, M2ST, NM2ST] > [CT], [DRT] > [M2ST]}; and (4) target load, [four target conditions of DRT, M2ST, NM2ST] > [two target conditions of DRT, M2ST, NM2ST]. To assess areas that exhibited delay activity (item 1) and target load effects (item 4) across all subjects, we entered the respective (first level) contrast images in a second-level group analysis (one-tailed t test). In this and all the aforementioned analyses, we used a minimal cluster-size criterion (k = 10 voxels) and a statistical threshold of p < 0.05 that was adjusted for multiple comparisons using the false discovery rate (FDR) or the familywise error (FWE) correction. The FWE correction was only applied when mapping subjects' regions of interest (ROIs). In all other tests, we used the FDR correction.
Region of interest analyses.
We selected all areas as ROIs that (1) exhibited consistent delay-related fMRI activity in all individuals and (2) showed consistent anatomical overlap across subjects (as revealed by the respective second-level analysis) (compare with supplemental Fig. S3, available at www.jneurosci.org as supplemental material). Importantly, this ROI mapping criterion did not bias the selection of ROIs in favor of any particular task component such as memory, attention, or motor preparation (also refer to Results). Next, in each individual and for each of these functionally defined ROIs, the normalized mean β weights of the various delay-period regressors were extracted for a 3-mm-radius sphere around the voxel exhibiting a local maximum t value for the delay-activity contrast. In particular, we analyzed β estimates for the (1) left and (2) right superior parietal lobule (SPL), (3) the left anterior intraparietal sulcus (antIPS), (4) left and (5) right dorsal premotor cortex (PMd), (6) the supplementary motor area (SMA), and (7) the left dorsolateral prefrontal cortex (DLPFC). The relative contribution of these ROIs to the processing of information throughout the delay was assessed by rank numbers. Such ranks were assigned to each of these ROIs within individuals depending on their maximum t value: rank 1 was assigned to a single subject's ROI with the highest maximum t value, whereas rank 7 was assigned to the ROI with the lowest maximum t value. Across subjects, the average ranks for the abovementioned areas 1–7 varied significantly (p < 0.001, repeated measures ANOVA) and amounted to 1.6 (left SPL), 1.6 (right SPL), 3.5 (left antIPS), 3.9 (left PMd), 5.3 (right PMd), 5.8 (SMA), and 6.4 (left DLPFC), respectively.
In a next step, we calculated ROI-based group statistics across the β values of individuals. Please note that this ROI analysis was independent from our ROI mapping criterion. Specifically, for each ROI, we performed a two-way repeated measures ANOVA with the factors condition (DRT, M2ST, NM2ST) and target load (two, four). We tested for sphericity (Mauchly's test) and adjusted the F statistic according to the procedure of Greenhouse and Geyser whenever the assumption of sphericity was met. If the factor condition was significant, we performed additional post hoc comparisons using paired t tests. Respective p values were adjusted according to the Bonferroni's procedure. As for our behavioral performance plots, the related figures report all significant post hoc comparisons, and the β plots refer to normalized data (for details about normalization, see above, Performance monitoring and analysis).
Finally, event-related time courses (ERTs) of signal intensities were extracted using the NOD Lab toolbox NERT4SPM (by Axel Lindner and Christoph Budziszewski; http://www.hih-tuebingen.de/en/sensorimotor-lab/nod-lab/). As for the GLM analyses, image time courses were high-pass filtered (cutoff period, 128 ms) and normalized by an estimate of baseline activity. This estimate was based on the mean image intensity −5 to −2 s relative to delay-period onset averaged across all experimental conditions, i.e., it reflects the overall level of fMRI activity at the very end of the fixation period. To generate individual ERTs of the fMRI responses, signal time courses were aligned to the onset of the delay period as specified by the GLMs. Because of an additional temporal jitter in our design, we were able to express the resulting time courses at a 1 s temporal resolution. The respective ERTs thereby represent an average calculated across the normalized mean of each subject's ERTs (for normalization, see above, Performance monitoring and analysis).
Results
Behavioral performance
To guarantee that fMRI activity during the delay period would solely reflect the differential contributions of the prospective and retrospective processes under investigation, we tested for any task-related differences in subjects' behavioral performance in terms of saccades, eye blinks, reaction times, error rates, and trackball movements, which were monitored throughout the scanning sessions. Representative examples of these measures are provided in supplemental Figure S1 (available at www.jneurosci.org as supplemental material). Importantly, the number of fixational saccades (Fig. 2A) as well as the frequency of eye blinks (Fig. 2B) did not significantly differ within the delay period of different trial types [rmANOVA2 with the following factors: condition, p > 0.05, not significant (NS); target load, NS; interaction, NS]. Furthermore, although there was a significant effect of experimental condition on subjects' error rates (condition, p < 0.01; target load, NS; interaction, NS) (Fig. 2C), these rates were statistically indistinguishable between the “memory-only” (M2ST) and the “prospective planning conditions” (DRT, NM2ST: paired t tests, NS, Bonferroni's corrected for multiple comparisons). In the control task, CT error rates were significantly smaller than in the DRT and NM2ST (paired t test, p < 0.05, corrected).
Behavioral performance. A, B, Estimating subjects' average behavioral performance during scanning allowed us to guarantee that delay-related fMRI activity would not be biased by any systematic differences in the number of fixational saccades (A) or in the frequency of eye blinks (B). Moreover, performance levels did not differ between the DRT, NM2ST, and M2ST as indicated by the share of error trials (C). However, error rates in the DRT and the NM2ST were significantly increased compared with the control task, as was expected due to the additional task demands (CT; *p < 0.05, **p < 0.01, ***p < 0.001, corrected). Finally, the reaction times for the manual responses are shown in D. Importantly, reaction times in the DRT were significantly shorter compared with the CT. This implies that subjects had used the delay period to prepare the required sequence of finger reaches. Longer reaction times were found in the NM2ST and the M2ST because both tasks involved additional match-to-sample comparisons. There were no significant differences between two-target and four-target conditions. Error bars in this and all other figures reflect the SEM, calculated across the normalized averages of individual subjects. Normalization was performed to account for our within-subject design. Toward this end, the between-subject variance in the data was eliminated (for details, see Materials and Methods).
Reaction times significantly varied across experimental conditions (rmANOVA2; condition, p < 0.001; target load, NS; interaction, NS) (Fig. 2D): compared with both CT and DRT, manual reaction times were significantly higher in the NM2ST and the M2ST (paired t test, p < 0.01, corrected). This was expected, because movement performance in both M2ST and NM2ST critically depended on an additional match-to-sample comparison before movement initiation. More importantly, compared with the CT, reaction times in the DRT were significantly lower by ∼200 ms (paired t test, p < 0.001, corrected). This implies that, on average, subjects had actually prepared the upcoming movement sequence during the delay phase of the DRT and thus were faster to initiate movements than in the CT, in which a response had to be performed ad hoc, i.e., without previous planning (Rosenbaum, 1980). Note, however, that the possibility exists that shortened reaction times could also derive from the fact that subjects need less time to elicit movements to memorized target locations. However, in a comparable task, others have failed to demonstrate a significant reaction time benefit when a reach had to be planned and performed to a memorized visual goal rather than to a visible target location (Gail et al., 2007), rendering the latter interpretation less likely.
Finally, during the delay period, we did not register any finger movement that survived the response threshold used for the calculation of our reaction time estimates. Moreover, there were no significant differences in subthreshold absolute movement velocity across conditions (rmANOVA2; condition, NS; target load, NS; interaction, NS) (refer to supplemental Fig. S2, available at www.jneurosci.org as supplemental material).
In summary, the analyses of the various behavioral measures imply that subjects had prepared the precued movement sequence in the DRT and that, apart from related differences in reactions times, there was no distinction between the DRT, the NM2ST, and the M2ST with respect to subjects' overt behavior throughout the delay and their level of performance.
fMRI correlates of retrospective visual memory and prospective planning
Brain activity was analyzed using an ROI-based approach, which we chose to best account for the anatomical variability across subjects. Specifically, by contrasting a linear combination of delay-period fMRI activity across all memory and planning conditions with the corresponding period of the control task, we were able to functionally define ROIs that showed significant sustained activity in each individual subject (p < 0.05, FWE corrected) (supplemental Fig. S3, available at www.jneurosci.org as supplemental material). This procedure ensured that the selection of ROIs was not biased by any particular task component, for instance, working memory in the M2ST or motor planning in the DRT. Across all subjects, significant delay-related activity was mapped bilaterally in PPC, namely the medial aspects of SPL next to the IPS. This activation further spread along the IPS and formed another pronounced cluster of significantly activated voxels in its most anterior portion (antIPS). Furthermore, there was a consistent activation in both left and right PMd as well as the SMA. Finally, delayed fMRI responses were also present in the DLPFC, mainly associated with an activation of the left middle frontal gyrus (MFG). Although some subjects also seemed to recruit their right MFG, we could not detect significant delayed activity in all of our subjects. A graphical illustration of these functionally defined ROIs is shown in Figure 3. The activity pattern rendered on a canonical brain surface depicts the statistical map that was obtained as a result of a second-level group statistic calculated across the aforementioned “delay contrasts.” Importantly, this group map perfectly resembles individual subjects' maps used to determine our ROIs in the first place (supplemental Fig. S3, available at www.jneurosci.org as supplemental material). Moreover, all ROIs depicted in Figure 3 are in good spatial correspondence with previously described parietal, premotor, and prefrontal foci that have been attributed to goal-directed reaching and pointing, to visuospatial working memory, or to both (DeSouza et al., 2000; Rowe et al., 2000; Simon et al., 2002; Astafiev et al., 2003; Connolly et al., 2003, 2007; Curtis et al., 2004; Medendorp et al., 2005; Volle et al., 2005; Curtis and D'Esposito, 2006; Beurze et al., 2007; Mars et al., 2008). Besides, our ROI within the left SPL overlaps with a prevailing locus of lesion in optic ataxia patients (Karnath and Perenin, 2005; Trillenberg et al., 2007). These patients have severe difficulties in reaching toward peripheral visual targets, i.e., they have difficulties in performing the same behavior that was required by our tasks.
fMRI activity related to motor planning and visual memory. Significant sustained fMRI activity related to prospective motor preparation and/or retrospective visual memory was consistently mapped in each subject's posterior parietal cortex (SPL, antIPS), PMd, SMA, and DLPFC. This is illustrated by the second-level activation map that is overlaid on a canonical brain surface. For each of the aforementioned ROIs, which were separately mapped in each individual subject (MNI coordinates refer to average xyz location across subjects; for individual subjects' maps, refer to supplemental Figure S3, available at www.jneurosci.org as supplemental material), as well as for each experimental condition, the respective fMRI signal time courses are depicted (average ± SE across individual subjects): DRT in green, NM2ST in gray, M2ST in yellow, CT in blue. Time courses are aligned to the onset of the memory delay, and the solid vertical lines indicate the average duration of this delay epoch. Broken vertical lines denote the onset of the cues in the four-target/two-target conditions at −2.5s/−1.5s, respectively. The average ± SE normalized β weights for the delay period are illustrated by the bars. ROIs that exhibited a significant influence of the factor condition (DRT, M2ST, NM2ST) on these β estimates are indicated (*p < 0.05, **p < 0.01, ***p < 0.001; two-way repeated measures ANOVA with the factors condition, target load, and interaction). Post hoc comparisons between conditions revealed significantly stronger levels of sustained fMRI activity related to prospective planning (DRT > M2ST or NM2ST > M2ST) in ROIs within posterior parietal cortex and dorsal premotor cortex (paired t tests, Bonferroni's corrected for multiple comparisons; levels of significance are indicated next to the respective β estimates). L/l, Left; R/r, right.
To compare the contribution of the various task components to the sustained fMRI responses within these functionally defined ROIs, we used two approaches. In a first approach, we extracted the fMRI signal time course within each ROI and for each individual subject using the NERT4SPM toolbox. For each of these regions and for each experimental condition, we calculated the average ERT of the fMRI response across subjects. The resulting ERTs are depicted in Figure 3, all aligned to the onset of the delay period. These plots do not account for the hemodynamic delay of the fMRI signal (∼5–6 s time-to-peak). As expected, all ROIs showed an increased level of fMRI activity in the memory-only condition (M2ST, yellow traces) and the putative planning conditions (DRT, green traces; NM2ST, gray traces) compared with the control condition (CT, blue traces). This difference in activity emerged immediately after cue presentation and persisted throughout the delay period, indicating an involvement of these regions in putative planning or retrospective memory. The long delay periods of our experiment thereby permit depiction of delay activity in the absence of any visual or motor response contributions. As further illustrated in supplemental Figure S4 (available at www.jneurosci.org as supplemental material), this holds true for the “late” delay period (∼10–15 s after delay onset), which we will describe in the following.
In most ROIs, the level of sustained activity varied across the DRT, the NM2ST, and the M2ST: late delay activities were the strongest in the movement-planning condition DRT, response amplitudes were the lowest in the memory-only condition M2ST, whereas intermediate levels of activity were obtained in the NM2ST. Importantly, this pattern of differential activity was most pronounced in the left SPL, the left anterior IPS, as well as the left PMd. Similar but less robust differences emerged for the corresponding cortical areas in the right hemisphere (ipsilateral to the effector) and for the SMA. Only the DLPFC showed no variation of the fMRI responses across conditions, implying an involvement of DLPFC in the maintenance of retrospective visual memory but not in prospective motor planning.
In a second analysis, we extracted the β values obtained for each of the delay-related regressors of our single-subject analysis and for each ROI. β values were expressed as estimates of the percentage signal change of the fMRI signal. As for the ERTs, β values were averaged across subjects. Furthermore, because β values provided a single estimate of the “strength” of the delay-related fMRI signal, they were used to perform ROI-based group statistics. β values for each of the experimental conditions and for each ROI are shown as bars in Figure 3. Note that the β estimates closely follow the relative order of the ERT amplitudes at the end of the delay period. The amount of signal change captured by the delay-period predictors as reflected by their corresponding β values slightly underestimated the actual level of fMRI activity during the delay. This was expected, because the regressors for the cue response already captured parts of this activity. More importantly, we obtained no evidence for a carryover of visual activity, influencing the β estimates of the delay period (compare with supplemental Fig. S4, available at www.jneurosci.org as supplemental material).
Subsequent statistical analyses revealed a significant influence of the factor experimental condition on the β values in bilateral SPL, bilateral PMd, as well as the left antIPS [rmANOVA2; factors of condition (DRT, NM2ST, M2ST), target load (two or four targets), and their interaction] (Fig. 3). A significant influence of the factor target load was revealed in all ROIs except for the DLPFC: the β estimates of the four-target conditions were significantly larger than those of the two-target conditions (Fig. 4). There were no significant interactions between the factors of experimental condition and target load.
Target load-related fMRI activity. fMRI activity related to target load is illustrated by the second-level activation map, which is overlaid on a canonical brain surface. Load effects were most pronounced in subject's posterior parietal cortex (SPL, antIPS), but also PMd and SMA showed significant effects. For each ROI as well as for each experimental condition, the respective fMRI signal time courses are plotted (average across subjects): two-target conditions are shown by broken lines, four-target conditions are shown by solid lines. In addition, the differences in percentage signal change between the β estimates for the four-target and the two-target conditions during the delay period (± SE) are depicted by the bars. Asterisks denote a significant load effect (two-way repeated measures ANOVA with the factors condition, target load, and interaction; *p < 0.05, **p < 0.01, ***p < 0.001). There was no significant interaction between condition and target load. The most pronounced differences in target load were obtained in SPL and in the DRT condition. Note that all ROIs and all additional figure conventions correspond to those of Figure 3. L/l, Left; R/r, right.
Next, we performed post hoc comparisons to identify specific, pairwise differences in β estimates between experimental conditions (paired t tests, Bonferroni's corrected for multiple comparisons) (Fig. 3). Specifically, β values for the DRT were significantly greater than those for the memory-only condition in all ROIs with a significant influence of the factor condition, namely SPL, antIPS, and PMd (for individual subjects' whole-brain maps of the same statistical contrast, see supplemental Fig. S3, available at www.jneurosci.org as supplemental material). However, in all of these ROIs, the normalized β values between the DRT and the NM2ST were statistically indistinguishable. Additional differences emerged between the NM2ST and the M2ST but only in the left SPL and the right PMd. This could imply a contribution of these areas to the inhibition of movements to undesired target locations.
Finally, we compared the β estimates between corresponding ROIs of the left and right hemisphere, namely bilateral SPL and PMd. Larger β estimates surfaced for the hemisphere contralateral to the effector, namely the left SPL and the left PMd (Fig. 3). Supplemental Figure S5 (available at www.jneurosci.org as supplemental material) provides a direct comparison of the respective β estimates and indicates statistical differences. Notably, in SPL, significant differences only emerged for the DRT and the NM2ST, highlighting a role of SPL in the prospective organization of upcoming behavior.
In summary, using variants of the delayed-response task, we found that fMRI activity in posterior parietal and dorsal premotor cortex cannot be explained solely by retrospective visual memory, which was identical across conditions. Instead, it indicates an additional involvement of these regions in the prospective planning of upcoming goal-directed movements while considering both types of targets relevant for action, those to be acquired (DRT) and those to be avoided (NM2ST). In contrast, fMRI activity within DLPFC rather reflects the maintenance of visual memory (Funahashi et al., 1989).
Discussion
Prospective planning in posterior parietal cortex
The role of PPC in motor planning is a matter of ongoing debate (Mishkin and Ungerleider, 1982; Goodale and Milner, 1992; Snyder et al., 1997; Colby and Goldberg, 1999; Andersen and Buneo, 2002; Andersen and Cui, 2009; Andersen et al., 2010). To date, most convincing evidence in favor of its role in planning stems from electrophysiological studies in monkeys that demonstrate effector-specific “movement intentions” in delayed-response tasks: sustained neuronal firing in functional subregions of PPC distinguishes between the planning of eye (lateral intraparietal area) versus reach (parietal reach region) movements to the same remembered target location (Snyder et al., 1997; Cui and Andersen, 2007): PPC neurons did not reflect mnemonic demands, which were identical, but rather the way in which a memorized target was acquired. Several groups adopted this approach in human imaging and demonstrated effector-specific delay activity in human posterior parietal and premotor cortex (Astafiev et al., 2003; Connolly et al., 2003; Medendorp et al., 2005; Beurze et al., 2007; Connolly et al., 2007), supporting their contribution to motor behavior. However, for detecting prospective planning for goal-directed movements, mapping effector-specific fMRI activity seemed not ideal. This is because effector-specific activity could also refer to variations in task difficulty or to differences in retrospective information that needs consideration when using eyes versus hands (such as proprioceptive information about the current state of the effector). Moreover, imaging effector-specific preparation may fail to elucidate prospective planning processes that refer to abstract, effector-independent aspects of behavior (e.g., planning a visual trajectory common to different effectors).
To extract fMRI activity that reflects the various levels of prospective movement preparation (ranging from visual trajectory planning and sensorimotor transformations to a specification of motor commands), we contrasted variants of the delayed-response task (DRT, NM2ST) with a memory-only condition (M2ST) that engaged the same effector and required the same amount of movement during the response, thereby the level of nonspecific movement preparation was matched. This matching is supported by identical levels of delayed fMRI activity in left motor cortex (supplemental Fig. S4, available at www.jneurosci.org as supplemental material) and by the same amount of residual movement during the delay (supplemental Fig. S2, available at www.jneurosci.org as supplemental material). Moreover, tasks were matched in terms of subjects' behavior during the delay (i.e., frequency of saccades and blinks), in terms of task difficulty as expressed by subjects' error rates (Fig. 2) and in terms of retrospective memory demands (see supplemental Discussion, available at www.jneurosci.org as supplemental material). However, compared with the DRT (and, potentially, to the NM2ST, which will be discussed below), the memory-only M2ST should not prompt any prospective planning processes during the delay phase because neither relevant nor irrelevant motor responses could be predicted based on the precued targets. Accordingly, we considered all areas that showed significantly increased levels of fMRI activity (compared with the M2ST) as regions engaged in prospective planning.
Prospective planning activity in DRTs was most pronounced in the PPC (SPL, antIPS) but was also present in premotor cortex (PMd). Moreover, in both SPL and PMd, this fMRI activity was significantly stronger in corresponding regions of the left hemisphere, contralateral to the effector (supplemental Fig. S5, available at www.jneurosci.org as supplemental material), supporting their contribution to motor behavior. The fact that, in SPL, this hemispheric difference would only emerge in DRTs and NM2STs further highlights the role of PPC in prospective planning.
Default planning versus retrospective visual memory in PPC
In most studies that tried distinguishing prospective movement planning from retrospective visual memory, PPC seemed to contribute to the maintenance of visual memory rather than to the planning of movement. Instead, movement planning was commonly attributed to premotor cortex. Partially, this view emerged from experiments that used memory tasks, in which subjects had to remember several potential cues while only later a second cue would specify the movement target (Rowe et al., 2000; Volle et al., 2005; Curtis and D'Esposito, 2006; Mars et al., 2008). Unfortunately, this approach bears the possibility that subjects do not only remember the spatial cues but also prepare all movements that are potentially required (Snyder et al., 1997; Cisek and Kalaska, 2002, 2005; Cui and Andersen, 2007). The fact that SPL (as well as PMd) may plan more than one movement at a time, as evinced by the modulation of sustained activity proportional to the number of response targets (Fig. 4), provides additional support for this notion (cf. Medendorp et al., 2006). Interestingly, in two of these memory studies (Volle et al., 2005; Curtis and D'Esposito, 2006), the specification of the final movement goal was followed by a second delay. In line with the idea of default planning, sustained PPC activity decreased in the second delay: this would be attributable to a lower number of movement plans represented after the response had been specified. Finally, in another experimental approach, Curtis et al. (2004) tried distinguishing prospective planning and retrospective memory by comparing fMRI activity in a delayed-saccade task with a delayed non-match-to-sample comparison similar to our NM2ST. As discussed in the next section, the NM2ST cannot control for memory-related processes because it does not rule out the possibility that subjects inhibit movements toward precued locations.
Note, however, that we do not claim that the aforementioned examples of PPC activity could solely be explained by prospective planning. In fact, here all parietal and premotor “planning areas” also exhibited elevated sustained fMRI activity in the M2ST compared with the CT. Although the nature of this residual fMRI activity remains to be revealed, it likely relates to the maintenance of retrospective visual information (Rowe et al., 2000; Simon et al., 2002; Curtis et al., 2004; Volle et al., 2005; Curtis and D'Esposito, 2006; Mars et al., 2008). Also, it might reflect subjects' arousal because of higher task demands (Fig. 2C) or covert shifts of attention to the precued locations (Kastner et al., 1999; Corbetta and Shulman, 2002).
Prospective motor planning in the NM2ST?
In the NM2ST, subjects had to remember a set of cues specifying undesired target locations and to perform movements toward a second set of random targets presented during the response phase. Importantly, subjects had to perform movements only to those targets that were not present in the first set. This procedure rendered subjects unable to predict the required movements already during the delay. Despite that fact, we detected increased levels of sustained fMRI activity in SPL and PMd in the NM2ST (compared with the M2ST), implying prospective planning processes. However, planning in the NM2ST might have simply referred to the “certainty” of movements performed to all remaining, potential targets, to a “default planning” of movements, or to covert shifts of attention toward these targets. We can dispel these interpretations, because activity in SPL and PMd increased with the number of undesired target locations (from two to four locations) (Fig. 4) but not with the number of default locations (seven to five, respectively) or with movement certainty (0.44 to 0.22). In addition, average movement certainty was identical (0.33) for NM2STs, M2STs, and CTs; therefore, we do not expect to see any differential fMRI activity attributable to movement certainty across those conditions. Rather, we speculate that increased sustained fMRI activity in NM2STs (compared with M2STs) reflects inhibition of movements directed toward the precued, undesired targets. We consider such inhibition a constituent part of prospective planning: despite the fact that it cannot elicit overt behavior, inhibition helps by actively reducing the number of response alternatives. This is relevant, because reaction times increase with the number of alternatives available (Hick, 1952; Pellizzer and Hedges, 2003) (note that we could not observe the same temporal benefit in NM2STs, because this task required an additional non-match-to-sample comparison, leading to longer reaction times compared with CTs).
Several groups have provided indirect evidence for inhibitory processes during goal-directed reaching and saccades (Howard and Tipper, 1997; McSorley et al., 2006). These groups have shown that the trajectory of movements with longer latency often deviates away from distractors (compared with trajectories in the absence of distractors), an effect interpreted through an incipient inhibition of distractor location in action-based representations (cf. Howard and Tipper, 1997), supposedly mediated by premotor cortex (Schlag-Rey et al., 1992; Burman and Bruce, 1997). Initial descriptions of this effect refer to movements immediately performed toward cues that appeared simultaneously with a distractor. Importantly, the same trajectory deviations also occur in precuing tasks with delayed responses and in situations in which distractors are expected to appear at the same time as the response target but fail to do so (Van der Stigchel and Theeuwes, 2006). This suggests that subjects are able to maintain the inhibition of an undesired action goal (or plan) throughout a delay and in the absence of that goal. Because in our NM2ST task subjects had to perform movements toward all targets except those shown in the cue period, such sustained inhibition of actions toward the precued, undesired locations seems a possible strategy. In fact, subjects reported that they remembered which targets to avoid. No subject reported a default planning of movements toward the remaining locations. The latter strategy also would have been more demanding with respect to memory load (seven/five compared with two/four targets, respectively). Thus, we propose that increased parietal and premotor fMRI activity in the NM2ST relates to a sustained inhibition of action goals. This interpretation is consistent with higher fMRI activity in the four-target than in the two-target condition. Unfortunately, we cannot provide direct evidence supporting this notion: because this result was unexpected, our design lacks conditions that would allow estimating inhibition-induced trajectory modulations or the latency benefits discussed above.
Human imaging studies on the cancellation of preplanned motor responses confirm a contribution of parietal and premotor cortex to response inhibition (Garavan et al., 1999; Watanabe et al., 2002; Curtis et al., 2005; Brown et al., 2006; Cavina-Pratesi et al., 2006). However, in these studies, there was interference between a pre-potent behavior and its subsequent cancellation, whereas here the inhibition of a goal-directed motor behavior potentially occurred in isolation (cf. Hasegawa et al., 2004; Snyder and Lawrence, 2004). To date, the only demonstration of neuronal responses related to such isolated inhibition of behavior stems from monkey electrophysiology. Hasegawa et al. (2004) show that premotor neurons in both frontal eye fields (FEFs) and pre-FEFs have sustained activity that specifies whether or not a monkey should look at a precued target location. Our results on the planning of finger reaches are consistent with their finding and imply an additional role of parietal cortex. Finally, the results of Hasegawa and colleagues could explain why we failed to detect differences in fMRI activity between DRTs and NM2STs: the respective populations of neurons (“look” vs “don't look” neurons) were spatially intermingled, making it difficult for conventional fMRI techniques to detect signal differences.
Conclusions
Our results clearly demonstrate that human PPC is critically involved in the prospective preparation of upcoming movements while coding for both types of targets relevant for action: those to be acquired and those to be avoided. Similar to the manner in which PPC highlights the utility of a planned behavior on the basis of expected reward (Platt and Glimcher, 1999; Musallam et al., 2004; Sugrue et al., 2004; Iyer et al., 2010), it could likewise signify the level of action rejection in case of undesired behavioral outcomes.
Footnotes
-
This work was supported by grants from the National Institutes of Health and the Boswell Foundation. We thank all members of the Andersen Laboratory for their support, in particular Alexander Gail and Hilary Glidden for stimulating discussions and Viktor Shcherbatyuk for computer support. Moreover, we are grateful for all assistance provided by the staff of the Caltech Brain Imaging Center: Shawn Wagner, Daniele Procissi, Steve Flaherty, and Mary Martin.
- Correspondence should be addressed to Axel Lindner at his present address: Hertie Institute for Clinical Brain Research, Department for Cognitive Neurology, Hoppe-Seyler-Strasse 3, 72076 Tübingen, Germany. a.lindner{at}medizin.uni-tuebingen.de
