Abstract
Traditional theories have considered decision making as a separate neural process occurring before action planning. However, recent neurophysiological studies of spatial target selection have suggested that decision making and motor planning may be performed in an integrated manner. It was proposed that multiple potential plans are concurrently formed and the ultimately selected action simultaneously emerges within the same circuits (Shadlen and Newsome, 2001; Cisek and Kalaska, 2010). In the present study, we recorded from the parietal reach region (PRR) and dorsal area 5 (area 5d) in the posterior parietal cortex (PPC) while monkeys performed a nonspatial effector (saccade vs reach) choice task. The results show that PRR encodes potential and selected reach plans whereas area 5d encodes only selected reach plans, suggesting a serial visuomotor cortical circuitry for nonspatial effector decisions. Thus, there appears to be a different flow of processing for decisions and planning for spatial target selection, which is more integrated, and nonspatial effector decisions between eye and limb movements, which are more serial.
Introduction
To produce appropriate actions in response to complex and dynamic environments, the brain must rapidly generate potential action plans based on perceptual representations of the external world, choose among them, and then translate the selected plan into a detailed motor command leading to a muscle activation pattern. Traditional theories posit that these neural processes are implemented sequentially by distinct perceptual, cognitive, and motor systems (Miller et al., 1960; Keele, 1968). In accordance with this serial model, only a single selected action is planned by the motor system after a decision is formed by the cognitive system. Nevertheless, accumulating evidence points to multiple potential plans being simultaneously represented in virtually all motor-related areas (Riehle and Requin, 1989; Platt and Glimcher, 1997; Basso and Wurtz, 1998; Cisek and Kalaska, 2005; Baumann et al., 2009; Klaes et al., 2011) and such movement representations change dynamically according to prior information of behavioral context (Rickert et al., 2009). This distributed representation suggests that the same neurons involved in motor planning also compute the movement decisions (Shadlen and Newsome, 2001; Cisek and Kalaska, 2010). From an evolutionary point of view, forming multiple default motor plans and choosing between them in parallel may not only benefit decision making, but may also reduce reaction times and thus have a survival advantage (Andersen and Cui, 2009; Cisek and Kalaska, 2010; Lindner et al., 2010).
The examination of potential anatomical overlap for decision making and motor planning has focused on spatial target selection. In this type of task, choosing between movements is always associated with choosing between stimuli; thus, it obligatorily involves spatial attention because the selected target attracts more attention, which in turn engages numerous brain regions (Desimone and Duncan, 1995). However, it remains unclear whether a similar process is used for other aspects of movement decision-making. Beside target selection, animals often choose among multiple actions or strategies to achieve the same goal. In a previous study in which rhesus monkeys autonomously chose whether to move the eyes or arm to acquire a target in the absence of direct sensory instructions specifying the effectors, both the lateral intraparietal area (LIP) and parietal reach region (PRR) showed vigorous responses when effector information was ambiguous, suggesting the formation of potential plans (Cui and Andersen, 2007). Consistent with this idea, activity was maintained at an elevated level in these areas only if their preferred effector was chosen for the movement; otherwise the activity returned to baseline. To determine whether effector decisions also follow this integrated model, or have an additional hierarchical component that codes only the selected plan downstream of decision circuits, we recorded single neuron activity from dorsal area 5 (area 5d) and compared the results with adjacent PRR. The results indicate that area 5d is not involved when the effector decision is being made but rather encodes the selected reach plans after the end-effector is unambiguously specified as the arm. This result supports a serial sensory-motor processing circuitry for nonspatial, effector decision making.
Materials and Methods
Experimental preparation.
Three male rhesus monkeys (Macaca mulatta) weighing from 8 to 12 kg participated in this study. Under isoflurane anesthesia, a holder for fixating the monkey's head and a search-coil for measuring eye movements were implanted first. During training and experiments the monkeys sat, with head fixed, in a primate chair and faced a board consisting of touch-sensitive buttons. There were two light-emitting diodes (LEDs, one red and one green, next to one another) in each button for displaying visual stimuli. In each trial, the monkeys' eye and hand behaviors were controlled by a LabVIEW program running on a real-time PXI platform (National Instruments). Monkeys were first trained to look at the button with the illuminated red LED and to touch the button with the illuminated green LED with their hand contralateral to the recording chamber. Next, they were trained in the memory tasks to move their eyes or hand to the memorized location where a peripheral red or green LED was previously illuminated (Snyder et al., 1997). Finally, they were trained to perform the effector instructed and effector choice tasks. Once monkeys learned all tasks, a craniotomy was performed over the intraparietal sulcus (IPS) and a recording chamber was implanted over the craniotomy. All procedures were in accordance with NIH guidelines and were approved by the Institutional Animal Care and Use Committee of California Institute of Technology.
Behavioral paradigms.
At the beginning of each trial, both the green and red LEDs in the central button were turned on for 800 ms, and the monkey was required to fixate and touch it to initiate a trial. Following this stage the animals performed one of 3 trial types.
In “effector choice” trials (Fig. 1A), both the red and green LEDs in a peripheral button were turned on together for 600 ms. The monkeys were required to keep fixating and continue touching the central button. After both peripheral LEDs were extinguished, the monkeys were still required to withhold the motor response for another 600 ms (delay period) until the offset of the LEDs in the central button (GO signal). In these trials, the monkeys were only instructed regarding a spatial goal. They were allowed to choose to either move the eyes to the location of the peripheral target while keeping the hand on the central button, or to reach to the peripheral target while continuing to fixate straight ahead. Since these trials were interleaved with effector instructed trials, the monkeys could not know whether they had choice options before the cue offset. A competitive algorithm was adopted to balance the monkeys' choice frequency between saccades and reaches (algorithm 1, Barraclough et al., 2004). Like the monkeys, the computer also decided between a saccade and a reach and the monkeys received a reward only if their effector choice matched the computer's choice. Since the computer tried to bias its choice frequency against the monkeys' choice frequency in the preceding effector choice trials, the monkeys learned that the optimal strategy for them was to choose to saccade and reach equally often. Consequently, this strategy not only maximized the monkeys' reward, but also balanced the monkeys' bias in saccade/reach selection and their reward expectation (Cui and Andersen, 2007).
In “effector instructed” trials (Fig. 1B), both the red and green LEDs in a peripheral button were first turned on for 600 ms, the same as in effector choice trials. After the 600 ms cue duration, either the red or the green LED disappeared while the other LED stayed on to instruct which effector to use. In half of the trials only the red LED stayed on, instructing a saccade after another 600 ms delay period (effector instructed saccade). The end of the delay period was also signaled by the disappearance of both the peripheral red LED and the central fixation spot (GO signal). In the other half of the trials, the red LED was turned off, and only the green LED stayed on, instructing a reach after a delay (effector instructed reach). In these trials, the spatial target and movement effectors were cued asynchronously so the monkeys knew the target location, but not the instructed effector, during the first 600 ms from cue onset. The animals received liquid reward for all trials in which the instructed movement was correctly performed.
In “memory” trials (Fig. 1C), only either the red or green LED in the peripheral button was turned on for 600 ms, instructing a saccade or reach after another 600 ms delay. The monkeys were required to continue fixating and touching the central fixation spot until it disappeared (GO signal), and then acquired the memorized target by saccading or reaching according to the previously cued color. In these trials, the spatial target and movement effectors were cued together so the monkeys could prepare a movement immediately after cue onset. The animals received liquid reward for all trials in which the movement was correctly performed.
Data collection.
During recording sessions, microelectrodes were lowered into area 5d and PRR, and guided by previously collected structural magnetic resonance images (MRIs). Recordings from area 5d were made on the gyral surface of the cortex next to the medial bank of the IPS (anatomically equivalent to the middle part of PE shown by Matelli and Luppino, 2001, their Fig. 1), identified by a strong somatosensory response by touching the hand or foot, and ∼1–3 mm below the dura. The PRR neurons were recorded from the medial bank of the IPS, corresponding to the anterior part of the medial intraparietal area (MIP) previously defined by Matelli and Luppino (2001), and ∼4–6 mm below the dura as an anatomically segregated cluster from area 5d.
Once a neuron was isolated, its response was examined with the memory task for reaches and saccades. If there was a significant response during the cue or delay period before the reaching movement toward a direction, then the recording sessions proceeded to the effector choice and effector instructed paradigms. Since the primary goal of this study is to examine effector selectivity instead of spatial selectivity, the preferred direction of a cell was qualitatively determined by the largest overall response during the cue and delay period. The target in the choice and instructed trials was pseudo-randomly chosen from two peripheral locations so that one was in the preferred direction and the other was in a nonpreferred direction shifted by 90–180° from the preferred direction. In each recording session, effector choice and instructed trials were pseudo-randomly interleaved with equal (50%) probability. For each isolated cell, ∼8–12 trials were recorded for each combination of target location and trial type.
Single-neuron recordings were made with a movable multiple-electrode array (3- or 5-channel mini-matrix, Thomas Recordings). The raw electronic signal recorded from each electrode was preamplified via a headstage (20×), and then band-passed (300–40,000 Hz) and amplified (1000–20,000×) by a Plexon data acquisition system and saved for offline sorting. Data on spike waveforms/timing, the monkeys' eye and hand positions, and displayed stimuli were automatically stored on a computer for post hoc analysis using Matlab 6.5 (MathWorks).
Results
Given cells in both PRR and area 5d exhibit persistent responses before delayed reaches (Kalaska and Crammond, 1995; Snyder et al., 1997), one may conceive that they fire in a similar fashion during effector choice and memory tasks. Surprisingly, our results demonstrate that area 5d exhibits a totally different pattern of activity from PRR when effector information is delayed with respect to the spatial cue in the effector choice compared with the memory task.
Figure 2 shows neuronal activity for an example area 5d cell while the monkey performed effector choice (A), effector instructed (B), and memory (C) tasks. In contrast to PRR (as shown by Cui and Andersen, 2007, their Fig. 2), area 5d did not show any significant response to the stimulus until the effector was unambiguously specified as the arm in both the effector choice (Fig. 2A) and instructed tasks (Fig. 2B), whereas the same neuron started to fire much earlier after target onset in memory trials in which the stimulus was cued as the reach target at the beginning (Fig. 2C). Therefore, the example area 5d neuron appeared not to encode potential reach plans while the effector decision was being formed, but instead carried only the decided reach plan once the arm movement was chosen or instructed.
Figure 3 plots the population average activity of 91 PRR cells (36 from monkey D and 55 from monkey C, same population as the PRR cells published in the work of Cui and Andersen, 2007) and 54 area 5d cells (34 from monkey C and 20 from monkey T). The target presented in the response field of the PRR cells evoked a strong response while the final motor effector was unknown/undecided (Fig. 3A,C). The activity started to diverge after the motor effector was chosen (Fig. 3A) or instructed (Fig. 3C). On the other hand, the visual stimulus did not trigger a significant response in area 5d unless it was unambiguously associated with a reaching arm movement (Fig. 3B,D). In the effector choice trials, both PRR and area 5d neurons exhibited slight reach selectivity prior to the cue offset (Fig. 3A,B), whereas such a bias was not present in the instructed trials (Fig. 3C,D).
To illustrate the responses of PRR and area 5d populations during different time intervals, Figures 4 and 5 plot average activity during cue and late delay periods versus baseline activity during effector choice (Fig. 4) and effector instructed (Fig. 5) trials. During the cue presentation, because effector choice trials were randomly interleaved with effector instructed trials, effector information was ambiguous and both a saccade and a reach were candidate motor plans. In this period, PRR cells showed a strong response (p < 10−15, Figs. 4A, 5A) while area 5d cells did not exhibit significantly elevated activity (p > 0.05, Figs. 4B, 5B) above the baseline. In contrast, both PRR and area 5d populations showed significant premovement activity during the late delay period regardless of whether a reach was chosen or instructed and whether the target was extinguished or stayed on (p < 10−8, Figs. 4C,D, 5C,D).
Figure 6 compares population averaged activity of area 5d cells in all three tasks: effector choice, effector instruction and memory. The three PSTHs converge before reaching onset, again consistent with the movement plan being decided early, for the memory task, or later for the choice and instructed tasks.
In summary, using effector choice and instructed tasks in which the spatial target and motor effector were asynchronously specified, we found that PRR formed potential reach plans in ambiguous situations and it remained active only if the monkeys decided to move the arm, whereas cells in area 5d only reflected reaching plans selected by upstream brain areas after the effector decision was made.
Discussion
It has been proposed that behavior results from a constant competition between potential actions (Kim and Shadlen, 1999; Cisek, 2007; Beck et al., 2008; Wang, 2008). In the effector choice task, if the deliberation is performed by a competition between potential saccade and reach plans, this competition might involve connections between LIP and PRR. LIP and PRR share the same (predominantly eye-centered) reference frame (Cohen and Andersen, 2002), which would benefit such a competitive computation. Of course, LIP and PRR might participate in decision formation in concert with frontal areas, such as the frontal eye field and premotor cortex. However, it is also possible that the decision is entirely formed in other higher brain areas and fed back to the PPC. If this is the case, PRR and LIP simply reflect the potential plans which are continuously updated; that is, these areas continue to fire after decision outcomes because only the selected plan is potential after the decision is made.
Our results are consistent with a recent study showing that area 5d is more involved in motor preparation and down-steam of higher cognitive areas in the intraparietal sulcus (Maimon and Assad, 2006). Such serial processing circuitry has also been observed while monkeys performed vibrotactile discrimination tasks, in which the primary and secondary somatosensory cortices form perceptual decisions while motor areas reflect decision outcomes (Romo and Salinas, 2001). In a similar effector choice between two hands, Hoshi and Tanji (2000) found that some neurons in the premotor area encode the forthcoming reach before the monkey being instructed which arm to use while other neurons were not active until the effector was specified. Such a decision between arms seems to involve the parietal cortex as well, because transcranial magnetic stimulation of the human PPC has been found to bias subjects' hand choice (Oliveira et al., 2010). Apparently, both kinds of effector choices involve the overlapping parietal-frontal circuits through strong interconnections between the two lobes (Strick and Kim, 1978; Wise et al., 1997). Nevertheless, the neural mechanism underlying the decision between eye and hand movement might be different from that for the decision between limbs, because the latter engages internal dynamics within the musculoskeletal system. Natural motor behavior is usually composed of coordinated movements of different body parts driven by coherent activation of muscle synergies (d'Avella et al., 2003) and may not be implemented in a strict effector-specific way (Heed et al., 2011). In contrast, the saccade/reach effector choice tasks allow conceptual action plans to be formed without immediate specification of concrete physical parameters controlling eye or body movements, providing a unique approach to elucidating cognitive and executive stages in the neural processes of decision making and movement planning.
Biased activity before effector decision formation
In the present study, the choice between saccade and reach was balanced by having the monkeys play a mixed-strategy game with the computer (Barraclough et al., 2004). In this behavioral context, the computer made choices according to the monkeys' choice sequence. Because no higher-order conditional probability was tested such as algorithm 2 of Barraclough et al. (2004), the monkeys could use a strategy (such as win-stay-lose-switch) to predict the computer's choice and maximize reward expectation. To discourage the monkeys from choosing a motor effector too early, we interleaved the instructed trials with the choice trials so the monkeys did not know the availability of options until the middle of the trials. However, this paradigm may still not have completely prevented the monkeys' from forming a bias toward the choice with a higher reward probability given the recent history of reward. This seems to be the case because the monkeys often received a reward in >50% (up to 58%) of trials. Consequently, both PRR and area 5d cells showed slight reach selectivity prior the cue offset in the choice trials (Fig. 3A,B). In contrast to the choice trials, such a bias was not present in the instructed trials (Fig. 3C,D). Because the competitive algorithm was only applied to the choice trials, the biased activity may have been averaged away as the effector was randomly specified later on by an instruction regardless of the previous choice sequence.
The early PRR activity is related to the reach plan
Given that the PRR neurons showed a strong transient response to the cue onset (Fig. 3A,C), one might argue that the early PRR activity before the effector choice is a purely sensory response. However, converging evidence suggests that the strong PRR response during the cue period is likely related, to a large degree, to a potential reach plan. First, previous studies have demonstrated that the transient PRR response lasts only for 100–200 ms and then decays rapidly if a saccade is planned (Snyder et al., 1997, 2000a,b), even when the visual target stays on all the time (see Pesaran et al., 2010, their Fig. 2). In the present study, PRR activity was maintained at a very high level throughout the entire 600 ms cue period until a saccade was chosen or instructed (Fig. 3A,C). Second, direct comparison of PRR activity in memory-guided and visually guided delayed reach tasks has demonstrated that most PRR cells (107 of 157; 68%) show little or no increase when the visual cue stays on during the delay (Hwang and Andersen, 2011). Third, even the initial transient PRR response has been found to be strongly modulated by the impending movement plan. During the anti-reach task, only a small fraction of PRR neurons (19 of 143; 13%) were visually tuned in the cue period whereas many more (64 of 143; 45%) encoded the motor goal exclusively (Gail and Andersen, 2006). Fourth, during reach target selection, the PRR response was attenuated shortly after the onset of both targets and well before the reach when a reach to the nonpreferred direction was chosen even though the target was still present in the RF (Scherberger and Andersen, 2007). Fifth, PRR neurons can even be activated when a reach is planned to an as-yet unspecified goal in the absence of any visual target in the response field (Calton et al., 2002). Sixth, before being instructed whether to make a movement to the remembered location of a flash target or in the opposite direction, PRR neurons were active for both potential plans in the delay period (Klaes et al., 2011). Since one of the potential plans was inferred and opposite the location of the cue, this result shows that activity can appear without a visual stimulus falling in the RF, and that PRR represents alternative potential movement plans before the context rule is specified.
Together, although the PRR response might encompass some early visual components, a large body of evidence indicates that PRR activity is strongly dependent on an impending reach plan. The present study demonstrates that the PRR response is persistent when a reach is a potential motor plan, but drops dramatically once reaching is no longer an option by either choice or instruction.
Serial versus integrated cortical processes for effector and target selection
The present study demonstrated that PRR can encode a potential reach plan before an effector decision is formed, and the selected reach plan thereafter, whereas area 5d activity reflects only the decision outcome after the effector is unambiguously specified as the arm. The absence of activity related to potential reach plans in area 5d suggests a serial cortical processing circuitry with motor preparation in area 5d being downstream to plan selection. As shown in our previously study (Cui and Andersen, 2007), the potential movements associated with different effectors (saccade vs reach) are planned in parallel in separated cortical areas (LIP vs PRR) before the decided action is further planned. However, in proceeding from PRR to area 5d, the planning becomes more serial. In the case of target selection, potential movements to different spatial targets appear to be planned in parallel within the same cortical areas across parietal and frontal cortices, including motor cortex (Riehle and Requin, 1989; Rickert et al., 2009), suggesting less of a hierarchy at the cortical level for this type of processing. Although the target selection ultimately must become serial as only the selected movement is specified as the behavioral output, the brain circuits exclusively encoding the selected spatial goal might exist at the very late stages because potential targets are simultaneously represented in the primary (Riehle and Requin, 1989; Rickert et al., 2009) and premotor cortex (Cisek and Kalaska, 2005), the primary motor output stages of cortex. Thus, there appears to be a different flow of cortical activity for decisions and planning for spatial target selection, which is more integrated/parallel, and nonspatial effector decisions between eye and limb, which are more serial.
Footnotes
This work was supported by the National Eye Institute and the James G. Boswell Foundation to R.A.A., and the Alfred P. Sloan Research Fellowship and the Whitehall Foundation to H.C. We thank D. Baldauf, E. Hwang, and G. Mulliken for helpful discussion during the course of this study; I. Kagan for help with MRI; K. Pejsa, and N. Sammons for help with animal handling and training; J. Baer and C. Lindsell for veterinary assistance; V. Shcherbatyuk for computer support; and T. Yao for administrative support.
- Correspondence should be addressed to He Cui, Brain and Behavior Discovery Institute, Georgia Health Sciences University, Augusta, GA 30912. hcui{at}georgiahealth.edu