Abstract
We investigated the dynamics of neuronal activity related to sensorimotor transformation during single experimental trials of a given stimulus–response (S-R) association task. A monkey was trained to perform wrist extension/flexion movements in the horizontal plane to align a pointer with a visual target while single unit activity in the primary motor cortex (MI) was being recorded. The stimulus was a colored light-emitting diode (LED) presented to either the left or right of a central reference point. The monkey had to point directly at the target (“compatible” S-R mapping) or point to the opposite side of the target position (“incompatible” S-R mapping), with the mapping rule specified by the color of the LED. Single neuron activities on the four correct trials (left/right stimulus × compatible/incompatible S-R mapping) were compared to determine whether such activities were more related to stimulus encoding and representation, to response preparation and execution, or to the “decision” processes translating the stimulus representation into a response representation. A novel mathematical technique, called LOCUS ANALYSIS, has been developed to quantitatively analyze and visualize the contribution of neuronal activity toward the sensory, motor, or sensorimotor (i.e., decisional) aspects of the task. Our data show that as a trial evolves, neuronal activity in MI, at a population level, is first correlated with the representation of the specific stimulus (the side of LED), then with the representation of the S-R mapping rule (the color of LED) as well as trial-specific S-R association (the conjunction of stimulus side and stimulus color), and finally with the representation of the behavioral response (extension or flexion wrist movement). Immediately after the issuance of the movement command, the populational activity in MI remains correlated with the trial-specific stimulus-response conjunctions, i.e., the context of the motor decision that the monkey has just made. Cells recorded successively in a single penetration tend to resemble each other in their pattern of firing on the four correct trials, suggesting a modular organization of neurons based on their functional role in the processing of the S-R association task. Our results indicate that MI belongs to a distributed network such that its neuronal activity reflects the underlying network dynamics that translate a stimulus representation into a response representation via the activation and application of appropriate S-R mapping rule.
- sensorimotor transformation
- decision
- primary motor cortex
- compatibility
- LOCUS ANALYSIS
- wrist extension/flexion
Trial-by-trial comparison of neurophysiological and behavioral measurements of an awake animal trained in a stimulus–response (S-R) association task allows insightful assessment of the role of single neurons in the animal’s perception and performance. Typically, the animal is required to perform certain S-R association to obtain a reward, and neuronal activity is recorded. Such activity may reflect the encoding and processing of the stimulus, the preparation and execution of the behavioral response, and/or the connection between the two within a specific trial of the task. To determine the functional role of a particular neuron in S-R processing, its firing activity recorded on each trial is scrutinized with respect to the exact behavioral context of that trial, i.e., the nature of the stimulus and the response and whether S-R association has been successfully made. In this way, neuronal activities related to the sensory, the motor, or more interestingly, the sensorimotor (“decisional”) aspect of the task can be assessed.
Previously, neuronal activities related to a perceptual decision was demonstrated in area MT (Newsome et al., 1989a,b; Britten et al., 1992) and area MST (Celebrini and Newsome, 1994, 1995) when a monkey was trained to discriminate the visual direction of a moving random-dot stimulus by making correct eye movement to appropriate locations. The experimental paradigm involved systematic manipulation of the strength of the stimulus to observe parallel changes in psychophysical judgment of the animal (in terms of signal discrimination threshold) and in the activity of single neurons (in terms of the probability distribution of average spike rate over the ensemble of trials). By applying techniques derived from the Theory of Signal Detection (Green and Swets, 1966), Newsome and his colleagues convincingly demonstrated a correlation of neuronal activity (in these visual cortical areas known to mediate visual motion processing) and perceptual decision on a trial-by-trial basis. An extension of this Theory of Signal Detection-based approach allows one to calculate the correlation of neuronal discharge to both stimulus categories and to response categories, therefore addressing the question of whether such neuronal activity is more related to the sensory or motor aspect of the task (Zhang et al., 1997).
In a different experimental paradigm involving oculomotor delayed response (Funahashi et al., 1993), monkeys were trained in a compound S-R association task in which, on some trials, deferred saccades were directed to the location of a visual target (prosaccade trials), whereas on other trials, saccades were made in the opposite direction (antisaccade trials). The pro- or anti-instruction on each trial was signaled to the monkey by the shape of the fixation cue (circle vs cross). On antisaccade trials, the monkeys learned to override the prepotent tendency to look toward the location of the remembered visual stimulus. The pro- and antisaccade trials were randomly interleaved within a block. Delay-period activity of the same neurons in the dorsolateral prefrontal cortex was analyzed using a two-way ANOVA (with stimulus side and saccade direction as factors). Stimulus-coding neurons (the activities of which were statistically significantly different only for stimulus side but neither for saccade direction nor for the interaction term) as well as response-coding neurons (the activities of which were statistically significantly different only for saccade direction but neither for stimulus side nor for the interaction term) were both found. They also provided some evidence for differential dynamics (transient vs sustained) of activities of the same neuron during the pro- and antisaccade trials.
The pro-/antisaccade paradigm is an example of the compound S-R association paradigm involving more than one S-R mapping rule for an animal to simultaneously acquire and apply on-line on each trial. This is different from other experimental paradigm for assessing the stimulus or response nature of neuronal response, e.g., that ofAlexander and Crutcher (1990), where (1) pro-/antitarget trials were blocked, and (2) the task involves goal representation in visually guided movement.
One distinct advantage of this pro-/antitarget movement paradigm is that the microstructure of a decision process can be examined. Note that with alternative S-R mapping rules, there must be (sensory) cells that intercept the auxiliary stimulus that serves as the cue for selecting an S-R mapping rule on any given trial (an example of this auxiliary stimulus is the shape of the fixation cue in Funahashi et al., 1993). The selected rule, in conjunction with the principle stimulus (target location in Funahashi et al.), determines an appropriate motor response. Analyzing the pattern of neuronal activities across different combinations of stimulus × response × mapping rule offers an opportunity to identify not only the neurons (or neuronal activity) that encode the primary stimulus and the motor response, but also those mediating the auxiliary stimulus and the associated S-R mapping rules. Another interesting question that can be addressed is how the ultimate fate of a trial is correlated with the neuronal activity accompanying and only accompanying specific S-R conjunction, and how the trial-by-trial generation of motor intent is dynamically related to the activation of appropriate S-R mapping rules (or inappropriate ones, which need to be suppressed).
In this paper, we adopt the same compound S-R association paradigm of pro-/antitarget movements to examine neuronal activities correlated with sensorimotor transformation. A monkey was trained to perform wrist rotation movements (either extension or flexion) in the horizontal plane in response to a visually presented target, and single unit activities were recorded from the primary motor cortex (MI). Previous studies have suggested the existence of three different types of units in MI: namely, input or sensory neurons that process relevant stimulus information; output or motor neurons that execute overt (behavioral) response; and interfacing or sensorimotor neurons that supposedly connect the two (Lecas et al., 1986; Riehle and Requin, 1989; Miller et al., 1992; Zhang et al., 1997). Different MI units had also been shown to encode either the position of a visual target (goal) or the direction of limb movement during movement preparation and movement execution in a visually guided arm movement task (Alexander and Crutcher, 1990). Here, we are interested in how neuronal activities in MI are related to the dynamic process of transforming the stimulus representation into a response representation via the activation and application of specific S-R mapping rules. A novel data analysis and visualization technique, called LOCUS ANALYSIS, is proposed to quantitatively determine the processing “locus” (namely, sensory, motor, or decision) of the activity of individual neurons. Our ultimate goal is to illuminate the microstructure and dynamical nature of neural activities related to a decision (sensorimotor transformation) process. Preliminary results in analyzing the data presented in this report (using a conventional, two-way ANOVA technique) have been presented elsewhere (Riehle et al., 1994).
MATERIALS AND METHODS
Material and experimental design. A male monkey (Macaca mulatta, 10 kg) was cared for and used according to the Guiding Principles in the Care and Use of Animals of the American Physiological Society. The animal held a pointer by a vertical handle and was trained to move the pointer in the horizontal plane by flexion and extension movements of the wrist. The axis of rotation of the pointer was exactly under the wrist. The pointer was enclosed by a concave, semicircular vertical panel and terminated 5 mm from the panel. Three vertical pairs of LEDs were mounted on the panel; one pair in the middle and one pair each 40° to the left and right of the central pair. The central pair consisted of two white LEDs, 1 cm apart, that were constantly on, marking the starting position. The other two pairs consisted of a yellow LED 1 cm above a blue one. These two side pairs served as the response signals (RS) and movement targets when they were on (cf. Fig. 1).
Experimental design and histology.A, A monkey was trained to perform wrist extension or flexion movements to align a pointer with a visual target (LED light) while single unit activities in MI were being recorded. The stimulus is a colored LED presented either to the left or right of a central point (“home position”). The monkey had to point directly at (“compatible” condition) or to the opposite side of (“incompatible” condition) the target position, depending on the color of the LED. B, Recording site reconstructed post mortem, as indicated by the shaded area. Thedashed line is the borderline between MI and premotor cortex. CS, Central sulcus; AS, arcuate sulcus; IPS, intraparietal sulcus; PrS, precentral sulcus; r, right; p, posterior. Scale bar, 2 mm.
To start a trial, the monkey had to align the pointer with the starting position and hold it there for 2 sec. Then, one of the four colored side LEDs went on. If the LED was yellow, he had to align the pointer with this LED (congruent mapping), and if the LED was blue, he had to align the pointer with the LED on the opposite side (incongruent mapping). The monkey were rewarded by a drop of apple juice only when movements were both fast and accurate. During recording sessions, the time criterion consisted of both the reaction time (RT), i.e., the time between the target onset and movement start, and the movement time (MT), i.e., the time between movement start and movement termination. To obtain the reward, the monkey had to perform the pointing movement directly toward the target position such that RT and MT did not exceed 0.5 sec and 1 sec, respectively. The accuracy criterion consisted of a window of 5° before and 10° after target center, within which the monkey had to maintain the pointer for 0.5 sec. During training sessions, the temporal windows and target widths were gradually reduced.
Trials for the congruent and incongruent mapping conditions were blocked during the training sessions and alternated from one session to the next. After the monkey had learned the two mappings, i.e., when a criterion of >80% correct was met in both conditions, he underwent surgery required for recording single-neuron activity. After surgery, the congruent and incongruent trials were randomly intermixed within each daily session, and single-neuron recordings started on the very first day of such intermixed sessions. During the recording of one neuron, at least 20 trials of each of the four types of trials formed by combining the two types of movement (extension and flexion) with the two mapping conditions (congruent and incongruent) had to be performed. Note that our paradigm differs from that of Alexander and Crutcher (1990), in which congruent and incongruent mapping conditions were blocked during recording sessions.
Surgical and recording techniques. After an initial training period of ∼4 months, a rectangular perspex chamber (inner dimension, 10 × 26 mm) was placed over MI of the right hemisphere, contralateral to the active hand, under halothane anesthesia (<0.5% in air). A mechanical device made it possible to fix the chamber, and thus the animal’s head, during recordings. Glass-insulated tungsten microelectrodes (impedance, 0.5–1.5 MΩ at 1000 Hz) were inserted transdurally within the cortex by controlling the vertical displacement with a hydraulic micromanipulator. The x–y position of the electrode was referred to a 20 × 16 coordinate system in 0.5 mm steps, which was then superimposed over the cortical surface after the animal was killed for histological control. The animal was anesthetized with ketamine (5 mg/kg, i.m.) and intravenous sodium pentobarbital and perfused through the left ventricle with 0.9% NaCl followed by 10% formalin. The brain was removed and suspended in formalin. Later, parasagittal sections (50 μm) were made from the block containing the electrode penetrations using a freezing microtome and subsequently stained with cresyl violet. This allowed us to reconstruct the location of electrode penetrations and to define the cortical areas. The boundary between MI and premotor cortex was defined on the basis of a cytoarchitectonic analysis of the density of the layer V giant pyramidal cells (Riehle and Requin, 1989).
A 486 microcomputer was used to control the LEDs and store the behavioral and neuronal data. These data consisted of the RT, MT, action potentials, and the mechanogram of the movement. The mechanogram was generated by a linear potentiometer coupled with the axis of the handle that was sampled at 500 Hz. The time between the occurrence of the RS and a 0.5° deflection of the output of the potentiometer was defined as RT, and the time between RT and the pointer stopping within 0.5° of the final target position was defined as MT. Raster displays of neuronal activity, as well as mechanograms, were available on-line on the computer screen. Off-line analysis of spike trains was performed by pooling individual correct trials of identical S-R configuration and contrasting the neuronal firing activities across these trial configurations, as discussed in detail in the next section.
Data analysis
Configuration of trials. In this compound S-R association task, there are all together eight (2 stimulus side × 2 mapping rule × 2 response side) possible outcomes; among those four are correct trials in which the monkey received the reward. There were relatively few error trials during the recording sessions, because the monkey had received extensive training. Therefore, our analysis concentrated on the four kinds of correct trials only. For notational convenience, we use subscripts (tags or “trial types”) 1, 2, 3, 4 to represent those types of correct trials throughout this paper (Fig.2): (1) trial type 1: stimulus side is left, mapping is compatible, response side is left; (2) trial type 2: stimulus side is left, mapping is incompatible, response side is right; (3) trial type 3: stimulus side is right, mapping is incompatible, response side is left; (4) trial type 4: stimulus side is right, mapping is compatible, response side is right.
Configuration of an experimental trial. Among the possible combination of stimulus × mapping rule × response, the four correct trials are selected for additional analysis; they are denoted as trial types “1,” “2,” “3,” and “4,” respectively.
Note that trial types 1 and 2, as well as types 3 and 4, share identical primary stimulus (stimulus side); trial types 1 and 3, as well as types 2 and 4, share identical response side; and trial types 1 and 4, as well as trial types 2 and 3, share identical mapping rule (or the auxiliary stimulus encoding those rules, i,e, the stimulus color).
LOCUS ANALYSIS. Let V1,V2, V3, andV4 denote the neural activity (e.g., firing rate within a selected time window) on trial types 1, 2, 3, and 4 (the four kinds of correct trials, as labeled above). Any neuron can be characterized by this pattern of firing activity (here “pattern” is used in reference to across-type comparison). To relate this firing pattern to the sensorimotor characteristic of a neuron, some theoretical observations will be made. First, let us consider the firing pattern of motor neurons. The activity of a pure motor neuron will be related to the nature of the motor response of the animal but not to the nature of the sensory stimuli that are delivered. Therefore, it would fire indiscriminately (i.e., fire with same rate) on trial types 1 and 3 and indiscriminately on trial types 2 and 4, but should differ in those two cases (V1 =V3 ≠ V2 =V4). We call this pattern of neuronal activity “purely response-related” (Fig.3a). (MI cells are known to be directionally tuned within the movement space. Because our design allows us to record neuronal firing associated with wrist extension or flexion movements only, it is theoretically possible that a pure motor neuron may respond with V1 =V3 = V2 =V4, as long as its preferred direction is perpendicular to the extension–flexion axis. In other words, a cell may still be a movement-related one even if its discharge during extension and flexion movements are identical. This is an inherent limitation of the 2 × 2 task design and our analysis.)
Schematic drawings of the pattern of neuronal firing rate across four trial types, denoted asV1, V2,V3, and V4, for different kinds of neurons. A, A pure response-related neuron, with V1 =V3 > V2 =V4 (top) orV1 = V3 <V2 = V4(bottom). B, A pure stimulus-related neuron, with V1 =V2 > V3 =V4 (top) orV1 = V2 <V3 = V4(bottom). C, A neuron related to S-R mapping-rule, with V1 =V4 > V2 =V3 (top) orV1 = V4 <V2 = V3 (bottom).D, A neuron, the differential firing of which is related to a certain S-R conjunction, in this case, to left stimulus and left response on compatible mapping trials (i.e., trial type 1), withV1 > V2 =V3 = V4(top) or V1 <V2 = V3 =V4 (bottom). Figure continues.
Next, let us consider the firing pattern of sensory neurons. In our case, there are two sensory attributes: stimulus side (the primary stimulus for S-R mapping) and stimulus color (the auxiliary stimulus specifying the particular mapping rule). [In principle, the monkey could also use vertical location of the LEDs (rather than their color) to determine the congruent/incongruent mapping rule, as the room is dimly lit. However, the small vertical separations of those target LEDs made them difficult to discriminate and, therefore, less likely to be used to encode the mapping rule (see Miller, Riehle, and Requin, 1992, in which the monkey had to discriminate the Go/No-Go rule using the position of the target). Even if vertical position was encoded, it would not change the nature of the following analysis.] Both attributes are necessarily confounded in the initial stages of visual processing but later become represented separately. A sensory neuron encoding the primary stimulus (i.e., stimulus location) would fire indiscriminately on trial types 1 and 2 when the stimulus is on the left, and indiscriminately on trial types 3 and 4 when the stimulus is on the right, but should differ in those two cases (V1 = V2 ≠V3 = V4). We call this pattern of neuronal activity “purely stimulus-related” (Fig.3b). A sensory neuron encoding the auxiliary stimulus (i.e., stimulus color) would fire indiscriminately on trial types 1 and 4 when stimulus color is yellow, and indiscriminately on trial types 2 and 3 when stimulus color is blue, but should differ in those two cases (V1 = V4 ≠V2 = V3). Of course, a neuron encoding the behavioral meanings represented by these colors, i.e., congruent S-R mapping when “yellow” and incongruent S-R mapping when “blue,” would also have this pattern of firing. We call this pattern of neuronal activity “purely rule-related” (Fig.3c).
In addition to the above firing patterns, there can also be neurons that fire to a particular conjunction of stimulus and response, for example, a neuron that fires discriminatingly only on trial type 1 (compatible mapping of a stimulus on the left) but not otherwise (V2 = V3 =V4 ≠ V1), a neuron that fires discriminatingly only on trial type 2 (incompatible mapping of a stimulus on the left) but not otherwise (V1 =V3 = V4 ≠V2), etc. We call this pattern of neuronal activity “purely conjunction-related” (Fig. 3d). Of course, because we are analyzing correct trials only, the neuronal activity could reflect either an S-R conjunction (the generation of a motor intent to a stimulus) or an S-S conjunction (the trial-unique combination of the primary and the auxiliary stimulus dimensions, e.g., side and color here). Both the “rule-related” and “conjunction-related” neurons belong to the generic class of stimulus-response “association” neurons that translate a sensory stimulus into a behavioral response.
The firing patterns shown in Figure 3 represent only idealized cases. More generally, an actual neuron is characterized by the quadruplet(V1,V2,V3,V4), the pattern of firing activity across the four correct types of trials. To exhaustively capture the processing locus of all neurons, we orthogonally decompose neuronal activities into the pure functional components (as defined above) by introducing variablesX,Y,Z (see for mathematical proof of uniqueness and completeness of this procedure):
Equation 1
Equation 2
Equation 3Variable X indicates the stimulus component (primary stimulus, i.e., stimulus side) of the neuronal activity,Y the response component of the neuronal activity, andZ the mapping-rule component (the auxiliary stimulus, i.e., stimulus color) with respect to the task. This is to say X(or Y,Z) reflects the contribution of the stimulus-side factor (or response-side factor, stimulus color factor) to the neuronal firing; it is the correlation of neuronal activity to the processing of stimulus (or response, mapping-rule) aspect of the task. Essentially, these variables are double subtractions (double contrasts) of neuronal activities across trial types with the particular stimulus or response conditions cancelled out for balanced design. For a pure stimulus-related neuron (i.e., a neuron related to the primary stimulus only), Y = Z = 0;X ≠ 0. For a pure response-related neuron,X = Z = 0; Y ≠ 0. For a pure rule-related neuron (a neuron related to the auxiliary stimulus, i.e., the mapping rule), X = Y = 0;Z ≠ 0.
To further extract the functional relevance of neuronal firing using the orthogonal (X,Y,Z) decomposition, we introduce the notion of “differential activity” (DA), operationally defined as the variance R2in the modulation of neuronal firing across all (the four correct) trial types:
Equation 4
The magnitude of R indicates how the neuronal activity is differentially related to the processing of trial-specific stimulus identification and response selection, e.g., the incongruent mapping of a left-side stimulus to a right-side response on a particular trial, as opposed to (perhaps) more general modulation of neuronal activity that is related to task performance (e.g., an overall arousal or readiness during a trial) but does not contribute to processing trial-unique information. The reason we set DA to beR2 rather than R is because the former obeys the χ2 statistics for the null hypothesis (i.e., the Vi values are random variables) that can be tested against for statistical significance (see , where DA is simply denoted as D). There, it has been shown that:
Equation 5So, we may introduce spherical coordinates (x,y,z):
Equation 6that have been properly normalized:
Equation 7The triplet (x,y,z) describes a point on a unit sphere, compactly representing the neural activity across four types of trials V1,V2, V3, andV4. The spherical loci are ±(1, 0, 0) for pure stimulus-related neurons, ±(0, 1, 0) for pure response-related neurons, ± (0, 0, 1) for pure mapping-rule neurons. Note that the plus and minus signs come as a pair, representing excitatory/inhibitory types of neurons, respectively. For instance, a neuron withV1 = V3 >V2 = V4 is mapped onto (0, 1, 0), whereas a neuron with V1 =V3 < V2 =V4 is mapped onto (0, −1, 0). Collectively, these six loci represent the set of primary loci for the task. Graphically, the two mapping-rule loci are on the north and south poles, whereas the pair of pure sensory loci and the pair of pure motor loci are arranged, at equal distance but alternating fashion, along the equator. The sphere is thus divided into eight equal partitions (octants) when these loci are joined by great circles (Fig. 4).
Spherical representation of neuronal activity pattern. The pattern of firing of a neuron across the four trial types, as described by a quadruplet (V1,V2,V3,V4), can be converted into a point on this unit sphere (with radiusR = 1) through Equations 1-3, 4, and 6. The location of the point (“spherical locus”) compactly characterizes the pattern of neuronal firing rate across the four trial types and hence its functional relevance in the processing of specific stimulus/response/associational aspects of the task. A neuron, the firing pattern of which belongs to one of the four “pure” kinds as described in Figure 3 will be mapped onto the fundamental loci on the sphere that serve as landmarks for interpreting neuronal activity pattern. Shown here are the four fundamental loci on the visible portion of the sphere (one of the eight octants), representing pure stimulus-related (S), pure response-related (R), pure mapping-rule related (r), and pure conjunction-related (H) neuronal activities. Note the dot labeled byH does not represent the origin of the coordinate system, but rather the conjunction-related locus, which is on the spherical surface equal distant to the S, R, and r loci.
In addition to the primary loci for pure stimulus-related, pure response-related, and pure mapping-rule neurons, there should beconjunction loci for neurons, the changes in activity of which are correlated with the association of a response with a stimulus on each and every trial. For example, a neuron that fires discriminately only on trial type 2 but not otherwise (V1 = V3 =V4 ≠ V2) is represented by the points ±(1, −1, −1)/✓3 on the sphere. Other conjunction loci are ±(1, 1, 1)/✓3 that discriminates trial type 1, ±(−1, 1, −1)/✓3 that discriminates trial type 3, and ±(−1, −1, 1)/✓3 that discriminates trial type 4 from other trial types. Again, they come in pairs, with one excitatory and one inhibitory. The eight conjunction loci are situated exactly on the center of one of the octants generated by the six primary loci. Together, the 8 + 6 = 14 landmarks form the set of fundamental(pure) loci on a sphere for this binary-valued, compound S-R association task with alternative mapping rules. When the neuronal vector (x,y,z) is plotted on a sphere with reference to these landmarks or pure loci, the functional role of a neuron (neuronal activity) may be revealed. The stimulus, response, and mapping-rule aspects of the task, as well as the very S-R conjunction that correlates with the trial-unique association of a specific response to a specific stimulus, are to be mediated by those neurons that occupy corresponding spherical loci. The mathematical basis of this procedure is described in the .
The magnitude of R, on the other hand, reflects to what extent neuronal activities V1,V2, V3, andV4 are different; i.e., how much the cell is involved in processing trial-unique S-R information. It is shown (in the ) that if V1,V2, V3, andV4 are random variables with identical mean and variance, then R2 obeys χ2statistics, a property that allows numerical tests for statistical significance of task-relatedness of the firing pattern of a neuron.This is the reason for the operational definition of DA measure to beR2 rather than R.
Neuronal data processing
The LOCUS ANALYSIS technique that we develop in this report can be applied as a temporal technique to study the moment-to-moment change of processing locus of a neuron. For this, all one needs are the perievent time histograms of firing probabilityV1(t),V2(t),V3(t), andV4(t) for different trial types. The implicit assumptions here are: (1) all trials in each trial type have identical temporal characteristics, or at least neuronal discharge is not significantly modulated by this trial–trial variability; (2) temporal characteristics of different trial types can be compared on a moment-by-moment basis, when the time is with reference to (i.e., properly “aligned” against) some common external event, such as the onset of the stimulus or the behavioral response. The goal of this temporal analysis is to reveal any consistent trend regarding temporal ordering of neuronal activities related to the processing of stimulus, response, S-R mapping rule, or S-R conjunction. For comparison, the more traditional methods for temporal analysis of motor cortical activities include the population vector-based approach, which depicted the evolution of a neuronal ensemble code (“population vector”) during a mental rotation task (Georgopoulos et al., 1989; Lurito et al., 1991) or during a drawing task (Schwartz, 1993), and the ANOVA-based approach, which correlated, on a moment-to-moment basis, the firing of individual neurons to different movement parameters (direction, distance, target position) based on a multivariate regression model (Fu et al., 1995).
For the present case, raw spike trains emitted by a neuron during individual trials of a neuron are displayed in the form of rasters, with trials being rank-ordered according to RT. Spikes were accumulated across the ensemble of trials (all of the same S-R configuration or trial type) in 20 msec bins to form peristimulus time histograms (i.e., individual trials aligned with respect to the time of stimulus onset, defined as the occurrence of the RS) or periresponse time histograms (i.e., individual trials aligned to time of movement onset). Figure5 gives examples of the activity of four single neurons with the commonly adopted representation.
Neuronal activities visualized in raster displays (top) and in perievent frequency histograms (bottom) for four different units. Eachdot in the raster corresponds to an action potential and each row a trial. The trials are rank-ordered according to RT, the shortest RT being at the top. The mean activity (averaged across all trials of a given type) is shown as frequency histograms, in 20 msec binwidth. Individual trial types are indicated by the numbers 1, 2, 3, and 4 (cf. Fig. 3). For each trial type are shown the peristimulus time histogram in which trials are aligned (time-locked) with respect to the onset of stimulus (indicated by RS,left) and periresponse time histogram in which trials are aligned (time-locked) with respect to the onset of movement (indicated by M, right).Horizontal tick marks, 100 msec; vertical tick marks, 10 impulses/sec. The black bar below each histogram indicates the mean movement duration. A, Cell Akbin 115 (cf. Fig. 7). B, Cell Akbin 126 (cf. Fig. 8).C, Cell Akbin 125 (cf. Fig. 9). D, Cell Akbin 123 (cf. Fig. 10).
To apply the LOCUS ANALYSIS technique as a tool for temporal analysis, the peristimulus (S-locked) and the periresponse (R-locked) time histograms were first smoothed. Individual spikes were accumulated across the ensemble of trials (all of the same type) using 1 msec bins first. The number of accumulated spikes divided by the number of trials (of that type) gives the firing probability at any particular millisecond (i.e., as a function of time t). This probability-density distribution of firing rate was then convolved with a Gaussian kernel exp(−t/(2ς2)) using a halfwidth ς = 20 msec, and finally the smoothed firing probability density was reaccumulated into 20 msec bins. This smoothing procedure was applied to spike histograms for all four trial types, starting from the time of stimulus onset (the occurrence of RS). For corresponding 20 msec bins, the four firing probabilitiesV1(·), V2(·),V3(·), V4(·) allow the X, Y, and Z values of that bin to be calculated according to Equations 1-3 and also the DA (R2 value) according to Equation 5. This was done for all 20 msec bins, starting from stimulus onset. Note that the R2 value (the DA measure) here should not be confused with the variance measure in more traditional statistical regression analysis (e.g., used in the the study of Fu et al. (1995)that correlates different movement parameters with a neuron’s moment-to-moment firing).
The histogram of R2 could be displayed to examine the time course of the DA of each neuron. The peak (maximum) ofR2 was picked, and if this peak value satisfied a statistical criterion at certain significance level (to be discussed below), it was recorded as a DA peak for additional analysis. Otherwise, it was excluded. Some peaks were spurious, in that they were really an accidental reversal in an otherwise clearly ascending or descending series of R2 values, and were therefore also excluded.
Because the average RTs across the four trial types are different, care was taken to ensure that no frivolous peaks inR2 are introduced simply because the bins inV1(·), V2(·),V3(·), V4(·) were somehow not “corresponding.” Stimulus- or response-locked time histograms of the functional components (X,Y,Z) are meaningful only for bins close to the reference bin (the bin representing the stimulus onset for the former and movement onset for the latter). Therefore, the selection of peaks in the S-locked analysis was constrained to [0, RT], and the selection of peaks in the R-locked analysis to [RT/2, 3RT/2], where RT is the average reaction time across all trials and all trial types. Outside this range, if a peak was present in both S-locked and R-locked analyses, it could then still be selected as a DA peak and further processed to determine its spherical locus; otherwise, it was discarded. Note that this restriction in peak selection will not all by itself lead to an inherent bias favoring the selection of stimulus-related peaks during [0, RT] and response-related peaks during [RT/2, 3RT/2]; the functional significance of a peak (i.e., related to stimulus or response) is to be determined by its spherical locus at the time whenR2 value peaks.
The (x,y,z) coordinates of DA peaks were used to determine the nearest loci the ascending-and-then-descending R2 intended to approach. Most times, the locus could be successfully determined, using a criterion of θ < θc = cos−1(0.888) (see for the rationale for choosing this criterion). Occasionally, the identity of a peak locus was difficult to determine, because it fell within the vacuum zones; those were deemed “unclassifiable.”
Peaks, as defined by maximums of R2, were also traced backward or forward bin-by-bin to find the bin number (“contact time”) corresponding to the closest spherical distance to the loci R2 had visited or was about to visit. This contact time defines the minimum θ so that the associated spherical coordinates capture the functional aspect of the upsurge of the differential neuronal activity during a trial. Often, at the bin of closest contact, maximum of R2 was achieved; however, it could quite likely be offset by one, two, or (occasionally) more than two bins. The contact time was recorded as well and served as the confirmation of the authenticity of a peak (minimum θ criterion). The spherical coordinates at closest contact were chosen in lieu of the original DA peak (by the maximum R2 criterion), provided that the R2 value then still reached statistical significance.
In forward or backward tracing of spherical coordinates, sometimes a peak (using the maximum R2criterion) could occur when the DA appears to be en route or transiting from one pure locus to another (“peak collision”). Spurious peaks arising as a result of the collision of (i.e., occurring in between) two successive authentic DA peaks were therefore excluded.
To examine the statistical significance of the DA peaks, theR2 value at the peak was submitted to a χ2 test (see , where R2value is also referred to as D value). To obtain the intrinsic neuronal variance (the value ς0 of the associated χ2 statistics), differential neuronal activity during the 1500 msec before the stimulus onset was also calculated (in 20 msec bins, but unsmoothed). Across a total 1500/20 = 75 bins, the mean and the variance of R2 (orD) for any particular neuron was calculated. The intrinsic neuronal variance ς0 could be estimated, from either the mean or the variance of R2. MaximumR2 values during a trial (after stimulus onset) had to exceed a significance level of p = 0.001 to be qualified as a DA peak. Because the peak R2values are calculated from the smoothed histogram, the effect of smoothing has been taken into account in the comparison of peakR2 values (after the stimulus onset) and the ς0 (evaluated from neuronal activity before stimulus onset) to determine the significance level of the peak.
RESULTS
The monkey participated in 27 recording sessions. Average RT is 336 msec (SD = 33 msec) for trials with the compatible mapping rule (trial types 1 and 4) and 374 msec (SD = 75 msec) for trials with the incompatible mapping rule (trial types 2 and 3). There is also a difference in average MT: 227 msec (SD = 28 msec) for compatible trials and 246 msec (SD = 35 msec) for incompatible trials. For individual trial types, the average RT is 356 msec (SD = 30 msec) for trial type 1, 409 msec (SD = 88 msec) for trial type 2, 339 msec (SD = 36 msec) for trial type 3, and 316 msec (SD = 24 msec) for trial type 4.
The activities of 154 cells were recorded in MI and processed off-line using the method of LOCUS ANALYSIS described above. The mean RT across the four trial types was taken to be 360 msec (with 20 msec binwidth). We first tested whether the DA measure indeed conforms to the predicted χ2 distribution when the pattern of neuronal activity across the four trial types are random variables (the gives proof of why this should be the case). For each cell, the mean and variance of DA during the 1500 msec (1500/20 = 75 bins)before the stimulus onset were calculated and displayed in a scatterplot (Fig. 6). Although there were large variations in the level of intrinsic noise fluctuations of each neuron, the population as a whole follow a linear relationship between the mean DA and its variance, with the best-fitting slope of k = 1.175. Theoretically, the mean and variance of a χ2-distributed random variable obeys a linear relationship with a slope k = 1.22 (for df = 3). The close match between the theoretical prediction and the data provided support for the usefulness of the DA measure of neuronal activity and for the validity of the proposed LOCUS ANALYSIS method. It also provided an estimate of the baseline DA value for each neuron, which is needed for testing the statistical significance of DA peaks during a trial (i.e., after stimulus onset).
A scatterplot of the mean and variance of the values of constructed DA measure (Equation 4) before the onset of the stimulus (background DA value). Each cell is represented by asingle square. Linear regression over the 154 cells (entire recorded population) reveals a slope of k = 1.175, thus demonstrating that the DA measure indeed conform to χ2(3)-distribution for randomly varying neuronal activity across the four trial types (which predicts a slope ofk = 1.22), despite intrinsic variation in neuronal noise that determines the actual DA value.
Example of distinct patterns of neuronal firing
As a trial begins, the DA value of a cell increases and reaches a peak, indicating the function-specific involvement of this cell in mediating the stimulus, response, S-R mapping rule, or S-R conjunction on any trial. The time histograms of R, as well as the histograms of X (stimulus component), Y(response component), Z (mapping-rule component), can be displayed to reveal the processing role of a neuron (Figs.7, 8, 9, 10). Figure 7 is an example of a neuron (same as in Fig. 5a) with pure stimulus (X) component, because the firing dynamics (when trials were time-locked to stimulus onset) were similar for trial types 1 and 2, in which the stimulus was on the left, and for trial types 3 and 4, in which the stimulus was on the right. The DA value (as well as the X value) increased monotonously to reach a peak at t = 160 msec (after stimulus onset), where peak spherical coordinates (0.980, 0.077, −0.184) specified the locus at θ = 11.5° from the pure stimulus locus (1, 0, 0) (see for calculations of pure loci and angular distance to a pure locus). Figure 8 is an example of a neuron (same as in Fig. 5b) with pure response component, because the firing dynamics (when trials were time-locked to response onset) were similar for trial types 1 and 3, in which the motor response was on the left, and for trial types 3 and 4, in which the motor response was on the right. The DA value (as well as theY value) increased monotonously to reach the peak att = −40 msec (before response onset), where peak spherical coordinates (−0.029, 0.997, −0.075) specified the locus at θ = 4.4° from the pure response locus (0,1,0). Figure9 is an example of a neuron (same as in Fig.5c), the firing dynamics of which (when trials were stimulus-locked) reflected predominantly the contribution of aZ (mapping-rule) component, because the cell increased its firing rate to reach a peak in trial types 1 and 4 but not in trial types 2 and 3. The DA reached the peak at t = 260 msec, where the peak spherical coordinates (0.174, 0.238, 0.956) specified a locus at θ = 17.6° away from the pure mapping-rule locus (0,0,1). Apart from the Z component, the activity of this cell contained some amount of X,Y, and therefore was not so “pure”; in fact, we found relatively few cells that were purely or predominantly related to S-R mapping rule from our entire sample. Finally, in Figure 10 is shown a neuron (same as in Fig. 5d), the activity of which contained equal amounts of X, Y, and Z component (here trials are response-locked). This pattern of firing occurred because the cell fired only in trial type 1 and was virtually silent in trial types 2, 3, and 4. Clearly, its neuronal activity was correlated with specific S-R conjunction (in this case, left stimulus and left response), with DA value peaking att = −80 msec (before response onset). Its spherical coordinates (0.562, 0.606, 0.564) specified a peak locus merelyθ = 2.0° away from the pure conjunction locus (1,1,1)/✓3 = (0.577, 0.577, 0.577). Of course, this pattern of neuronal activity could also be interpreted as being correlated to the specific S–S conjunction (in this case, stimulus at left side with yellow color).
A pure stimulus-related neuron. The DA, measured by R, is contributed to byX (stimulus)-component only throughout the course of the trial (here all trials have been locked to stimulus onset). There is hardly any contribution from Y (response)- orZ (mapping-rule)-component. Here, the horizontal axis represents time, with the left dotted line representing the time of stimulus onset, and the right dotted linerepresenting the time of response onset averaged across the compatible trials or incompatible trials separately (the two average values happen to be same for this cell).
A pure response-related neuron. The DA, measured by R, is contributed to by Y(response)-component only throughout the course of the trial (here, all trials have been locked to response onset). There is hardly any contribution from X (stimulus)- or Z(mapping-rule)-component. Here, the horizontal axis represents time, with the two left dotted lines representing the times of stimulus onset (with respect to the fixed response onset), averaged across the compatible trials or incompatible trials, respectively, and the right dotted line the time of response onset.
A rule-related neuron. The DA, measured byR, is primarily contributed to by Z(mapping-rule)-component, although there appears to be some contribution from X (stimulus)- and Y(response)-components as well. Here, all trials have been locked to stimulus onset, represented by the left dotted line. The two right dotted lines represent the response-onset time, averaged across compatible trials or across incompatible trials, respectively.
A conjunction neuron. The DA, measured byR, is contributed to equally by X(stimulus)-, Y (response)-, and Z(mapping-rule)-components. Here, all trials have been locked to response onset, represented by the right dotted line. The two left dotted lines represent stimulus-onset times (with respect to the referenced response-onset time) averaged across compatible or incompatible trials, respectively.
Proportion of DA peaks of each distinct type
Among the 154 recorded cells, 136 cells had at least one statistically significant DA peak (at p = 0.001 level), when DA (R2 value) was constructed from either the peristimulus time histogram (i.e., trials were time-locked to stimulus onset or S-locked) or periresponse time histogram (i.e., trials were time-locked to response onset or R-locked) of neuronal activities. The remaining 18 cells had neither stimulus-locked DA peaks nor response-locked DA peaks that reached statistical significance.
When neuronal activities were averaged by aligning trials to stimulus onset (stimulus locking), 121 cells had at least one DA peak, and a total of 212 peaks were identified. The spherical locus of those peaks allowed a classification (using a criterion angle of θc = cos−1(0.888) = 27.4°) into four categories based on their affinity to the 14 primary loci: those related to stimulus side, to response side, to S-R mapping rule, to S-R conjunction (see Data Analysis for details). The pie (Fig. 11) gives the proportion of DA peaks in each category. Note that the breakdown of the pie chart is according to peaks, not cells; a single cell might give rise to one or more peaks during the period of [0, RT] and, therefore, contribute to one or more categories. For those cells with multiple peaks, there does not appear to be any consistent trend regarding the nature of, or the transition between, peak categories.
Proportion of DA peaks that are related to stimulus, response, S-R mapping rule, and S-R conjunction when all trials are time-locked to stimulus onset for carrying out the analysis. The categorization of a DA peak is based on its affinity to one of the four kinds of pure (fundamental) loci, with a criterion of angular deviation of θc = cos−10.888 = 27.4°. See for more details of this classification scheme.
When the same ensemble of neuronal spike activities was averaged by aligning trials to response (movement) onset, 134 cells had at least one DA peak, and a total of 223 peaks were identified. The category of those peaks could also be classified and then represented by a pie chart (Fig. 12). The pattern of peak distributions across the categories is similar to that of Figure 11 in which trials are stimulus-locked. Note that there are relatively few peaks related to the S-R mapping rule compared with peaks related to stimulus side and response side. The apparent abundance of peaks related to S-R conjunctions, on the other hand, may be attributed in part to the fact that on the unit sphere, there are eight pure S-R conjunction loci, compared with the two pure stimulus loci, two pure motor loci, and two pure mapping-rule loci (Fig. 4, in which only the first octant of the sphere is shown). Thus, there is a fourfold difference in single units needed to explicitly encode the S-R conjunction information, compared with neurons that explicitly encode stimulus, response, or mapping rule.
Proportion of DA peaks that are related to stimulus, response, S-R mapping rule, and S-R conjunction when all trials are time-locked to response onset for carrying out the analysis. For classification of peaks, see legend to Figure 11.
Dynamics of DA peaks of each distinct type
Next, for each category (in the above pie charts), we looked at the distribution of the time of occurrence of peak DA across the duration of a trial. The distribution of stimulus-related and response-related peaks was displayed in the same histogram using a binwidth of 40 msec and plotted in Figure13a (in which trials are stimulus-locked) and b (in which trials are response-locked). Clearly, the stimulus-related peaks occur earlier than the response-related peaks, demonstrating a transition from a more stimulus-related representation to a more response-related representation in MI activities during a trial. This is true in the LOCUS ANALYSIS of both the stimulus-locked histogram and the response-locked histogram. Interestingly, the distributions for both the stimulus-related peaks and response-related peaks are tighter in the response-locked analysis than in the stimulus-locked analysis. This suggests that the stimulus-related neuronal activity in MI may be not directly related to the encoding of the physical attributes of the stimulus per se, but rather related possibly to the processing of the behavioral meaning represented by such a stimulus in the context of an S-R association task.
Distribution of the occurrence of stimulus-related DA peaks and response-related DA peaks as time progresses during a trial. The ordinate represents the percentage of peaks of each category normalized against the total number of peaks of that category. The abscissa represents time bins of 40 msec, when trials have been time-locked either to stimulus onset or to response onset for carrying out the analysis. A, All trials are time-locked to stimulus onset (0 msec), so that bin 1 stands for 0–40 msec, bin 2 for 40–80 msec, etc., after stimulus onset.B, All trials are time-locked to response onset (0 msec), so that bin −1 stands for −40 to 0 msec, bin −2 for −80 to −40 msec, etc., before response onset. Bin 0 here stands for 0–40 msec after response onset.
Apart from the DA peaks related to the primary stimulus or the behavioral response, the distribution of DA peaks related to the S-R mapping rule and to the S-R conjunctions was plotted as cumulative probability distributions in Figure 14a (in which trials are stimulus-locked) and b (in which trials are response-locked). The two distributions almost match each other in both the stimulus-locked analysis and the response-locked analysis, indicating that the representation of the S-R mapping rule and S-R conjunctions are dynamically related to neuronal activity in MI and followed approximately the same time course during a trial. Note that because of trial-by-trial variation in RT, a meaningful comparison of the two curves should be restricted to bins immediately after the stimulus onset (in the case of stimulus-locked analysis) or bins immediately preceding the response onset (in the case of response-locked analysis).
Cumulative distribution of the occurrence of DA peaks related to S-R mapping rule and to S-R conjunction as time progresses during a trial. The ordinate represents the cumulative percentage (percentile) of peaks of each category normalized against the total number of peaks of that category. The abscissa represents time bins of 40 msec in duration, when the trials have been time-locked to either (A) stimulus onset or (B) response onset. For meaning of bin numbers, see legend to Figure 13.
To compare the different time courses in the representation of the stimulus, the S-R mapping rule, the S-R conjunctions, and the response by single neurons in MI, the cumulative distributions of DA peaks were all displayed in one plot (Fig. 15). Here, the distribution of stimulus-related peaks is based on stimulus-locked analysis, the distribution of response-related peaks is based on response-locked analysis, and the distributions of peaks related to S-R mapping rule and S-R conjunctions are based on an average of the stimulus-locked analysis and response-locked analysis; the onset of the stimulus and the response during a trial is separated by an amount equal to the mean RT (which is taken to be 360 msec). This pattern of data demonstrates that MI activity during a trial is first related to the primary stimulus (i.e., stimulus side), then to the S-R mapping rule (auxiliary stimulus or stimulus color) and to trial-specific S-R conjunction, and finally to the behavioral response (response side). In other words, neuronal activity in MI is correlated with the psychological processes responsible for sensorimotor transformation (stimulus → decision → response) in this S-R association task.
Cumulative distribution of the occurrence of DA peaks related to all four categories, namely, stimulus, response, S-R mapping rule, and S-R conjunction, with data from stimulus-locked analysis and response-locked analysis combined. Here, the time between stimulus onset and response onset is taken to be 360 msec (the grand average of RT of all trials), so that stimulus-locked bins and response-locked bins can be properly aligned for comparison. The data of stimulus-related peaks are taken from stimulus-locked analysis, the data of response-related peaks from response-locked analysis, and the data of rule- and conjunction-related peaks are both averages of those from the stimulus-locked analysis and the response-locked analysis. The cumulative percentage of peaks (on the abscissa) has been normalized with respect to total number of peaks of each category.
Populational dynamics of sensorimotor transformation
One possible criticism of the above analysis is that our pattern of results might be dependent on the criterion used for categorizing DA peaks, i.e., one might wonder whether a stricter or looser criterion for the assignment of “unclassifiable” peaks might affect the outcome. Therefore, we make use of the spherical coordinatesx,y,z (discussed in Data Analysis) to represent all DA peaks on the unit sphere. This sphere, in the X-Y-Z space with a radiusr = 1 (cf. Fig. 4), is a compact representation and direct visualization of the functional decomposition of neuronal activity into the stimulus (X), response (Y), and mapping rule (Z) aspects of an S-R association task. After normalization according to Equation 6, these functional components are captured by the spherical locus (x,y,z). The loci for a purely stimulus-related neuron, purely response-related neuron, purely S-R mapping-rule neuron, and purely S-R conjunction neuron (the so-called fundamental loci) are landmarks on the sphere that serve as references for interpreting the DA of any particular neuron. Figure16 represents, in successive time frames, all DA peaks (of the entire population of recorded neurons) that had been identified in the stimulus-locked analysis (peaks of all categories in the pie chart of Fig. 11 are represented here, including the “unclassifiable” ones). Here, each frame of the movie (a–f) represents 60 msec of a trial, starting from the stimulus onset. DA peaks for the entire neuronal population are represented at appropriate spherical locations. Furthermore, for better visualization, the x,y,z coordinates have been properly reflected into the first octant of the sphere (without changing their absolute value or functional meaning). With reference to the set of fundamental loci, it can be seen that the population activity in MI (as described by the total number of DA peaks) starts to rise at ∼60-120 msec after the onset of the stimulus (occurrence of RS); the population activity is primarily related to the processing of stimulus side (frames a and b). The population activity then migrates (frame c) to the spherical locus related to the processing of trial-specific S-R conjunction as the representation of the S-R mapping rule becomes available. At ∼180-240 msec after stimulus onset, the majority of DA peaks are related to the S-R conjunction and to the response aspect of the task (frame d). MI population activity remains related to those aspects of the task at 240–300 msec, whereas the amount of stimulus-related DA peaks subsides (frame e). Finally, at ∼300-360 msec, neuronal activity in MI is predominantly related to the trial-specific S-R conjunction (frame f). This is at approximately the time when the monkey’s overt movement is initiated (RT averaged across all trials is ∼360 msec when response onset is defined with respect to extensor or flexor movement with a criterion of 0.5° deflection of the mechanogram). Because it took ∼50-100 msec (Schwartz et al., 1988) for the command from MI to result in an effector movement, it is at frame e (not framef) that motor programming has been completed. The DA peaks related to the trial-unique S-R conjunction in frame freflect neural processing after the motor command left the MI (at approximately frame e).
Spherical distribution and evolution of DA peaks as time progresses during a trial. Here, for successive time frames, the spherical loci of all identified DA peaks (from Fig. 11, including the “unclassifiable” ones) are properly displayed onto the visible portion of the sphere, where the fundamental landmarks or pure loci of the sphere are given by Figure 4. Each DA peak is represented by only one black dot at an appropriate location (as determined by the peak spherical locus) and at an appropriate time frame (as determined by the peak contact time, see Materials and Methods for details). All trials have been time-locked to stimulus onset, so that each successive frame represents a time lapse of 60 msec thereafter.A, Between 0 and 60 msec; B, between 60 and 120 msec; C, between 120 and 180 msec;D, between 180 and 240 msec; E, between 240 and 300 msec; F, between 300 and 360 msec.
Modular organization of distinct patterns of neuronal activity
The above results suggest that MI neurons are involved in a dynamic network responsible for the transformation of the stimulus representation into a response representation during a trial, with distinct neurons differentially related to the stimulus, to the response, or to the decisional aspects of a task. A natural question is whether these functionally different neurons are clustered in MI. We observed that neurons recorded successively in a single penetration tend to resemble each other in their pattern of firing histograms across the four trial types. Two examples of such sequence are shown in Figure 17. The gradual change of the firing pattern from one neuron to the next in this series strongly suggests a modular organization of MI neurons into functionally connected groups (cell assembly) that mediate different aspects of sensorimotor transformation. These data are consistent with the idea that MI neurons participate in a dynamical network that is widely distributed (possibly across many cortical areas) and having distinct, task-specific functional components.
Firing patterns, in terms of X(stimulus)-, Y (response)-, and Z(mapping-rule)-components and their temporal dynamics of successively recorded neurons in a single electrode penetration. Note the similarity and gradual change of the firing patterns across those neurons.A, A sequence of five neurons. B, Another sequence of four neurons. Here, all trials have been locked to response onset, represented by the right vertical line. The twoleft vertical lines represent the time of stimulus onset (with respect to the referenced response-onset time), averaged across compatible or incompatible trials, respectively. On some occasions, the two lines coincide.
DISCUSSION
The question we addressed in this study is the functional components and the dynamics of neuronal activity related to sensorimotor transformation during single trials in a given S-R association task. With a novel data analysis technique, the LOCUS ANALYSIS, we were able to approach this issue by decomposing the pattern of neuronal firing under different combinations of S-R conditions (“trial types”) into a three-dimensional vector space describing the stimulus, the response, and the mapping-rule aspects of the task. Our data show that as a trial evolves, neuronal activity in MI, on a population level, is first correlated with the representation of the specific stimulus, then with the representations of the S-R mapping rule and trial-specific S-R conjunction, and finally with the representation of the motor response. Therefore, MI activity is dynamically related to the sensorimotor processes that connect the stimulus-encoding stage with the response-production stage, by activating the appropriate S-R mapping rule and selecting an appropriate response.
This dynamical transformation of MI activity from a stimulus-related representation to a response-related representation, as reported here, is closely related to the previously demonstrated rotation of neuronal population vector in MI (Georgopoulos et al., 1989; Lurito et al., 1991). In the paradigm of Georgopoulos and his colleagues, a monkey was trained to move its arm in a direction perpendicular to and counterclockwise from the direction of a target light that changed in position from trial to trial. The activities of MI cells recorded during the performance of that task were collectively represented by a neuronal population vector in the three-dimensional movement space (Georgopoulos et al., 1983, 1986, 1988; Schwartz et al., 1988). It was shown that during RT, this population vector gradually rotated counterclockwise from the direction of the light to the direction of the movement. Although the population vector constructed by Georgopoulos et al. refers to the encoding by the ensemble of neuronal activities of a parameter related to the movement direction in the three-dimensional movement space (which is different from the X-Y-Z vector constructed in this report), its gradual rotation during a trial clearly indicates that neuronal activity in MI, at a population level, transforms from a stimulus representation (the representation of light direction) to a response representation (the representation of movement direction) during each trial of that task. Our results, therefore, are consistent with those of Georgopoulos’ mental rotation paradigm. Both support the conclusion that MI is dynamically involved in the representations of stimulus and response during sensorimotor transformation. One might argue that the early, stimulus-related representation is in fact an “automatically” activated response representation (motor program) that always accompanies the presentation of the primary stimulus and that is only being aborted later if it is not congruent with the correct movement (Kornblum et al., 1990; Kornblum, 1992). Our data, as those of Georgopoulos et al., are not able to support or discount this automatic activation interpretation.
One interesting finding about the dynamics of neuronal activity in MI is that immediately after the issuance of the motor command, the firing pattern of most MI neurons (as represented by the peaks in DA) changes from the encoding of a response to the encoding of the S-R conjunction (see Fig. 16f). This indicates that MI neurons, or the cortical network in which MI participates, retain the information about the trial-unique categories of both stimulus and response, even though a motor response has been selected and produced. This “postdecision” representation of both the decision and the context of such decision is important for the computation of reward expectancy and the detection of any possible change in reward contingencies. It could result from a self-monitoring system that would be useful for detecting errors, as revealed by event-related potential studies (Gehring et al., 1993; Falkenstein et al., 1995).
Our observation that cells recorded successively in a single penetration tend to resemble each other in their dynamical firing pattern across trial types suggests a modular organization for MI neurons, an organization based on their functional role in cortical information processing, as proposed by Szentagothai (1978) andMountcastle (1979). Both authors emphasized that all cortical areas are constituted of aggregates of similar anatomo-functional units. Essentially built to perform the same basic operation, neurons with similar processing roles—for instance, in the processing of the S-R association task—are in close proximity and form locally interconnected groups or ensembles. The neuronal ensembles are the smallest units of representation of the stimulus, the response, and the transition from one to the other. These dynamically coupled ensembles of simultaneously active neurons are widely distributed over different cortical areas (cf. Braitenberg, 1978; Requin et al., 1988; Gerstein et al., 1989; Riehle et al., 1996). Previously, it was reported that preferred directions of MI cells tend to cluster in columns (Georgopoulos et al., 1984), suggesting a modular organization based on the coding of movement parameters by the motor cortex. Here, we extend this suggestion of modularity and ensemble activity to the dynamic operation of the motor cortex as well. Although the neuronal ensembles themselves may be part of a more widely distributed, intercortical network, they are the smallest functional units of such a network for representing any of the behaviorally meaningful components: the stimulus encoding stage, the response production stage, and the transitional or decisional stage in between (Requin et al., 1992). Indeed, neuronal activity associated with sensorimotor transformations has been reported in prefrontal cortex (Di Pellegrino and Wise, 1993;Funahashi et al., 1993), premotor cortex (Rizzolatti et al., 1988;Riehle and Requin, 1989; Di Pellegrino and Wise, 1993; Crammond and Kalaska, 1994), MI (Riehle and Requin, 1989; Miller et al., 1992;Riehle et al., 1994), and area 5 (Seal and Commenges, 1985; Crammond and Kalaska, 1989; Seal, 1989) and area 7 (Andersen et al., 1987;Andersen, 1989) of the parietal cortex. Our data here further argue that the sensorimotor transformation (or decision) processes involve the activation of S-R mapping rule, the application of such mapping rule to generate S-R conjunction (or a motor intent), and the transformation of such intent into a motor program. Although MI might not be the site for the original source of these microstages of sensorimotor transformation, the activity of cell assemblies in MI nevertheless reflects dynamically these psychological stages of information flow. In this way, information processing at behavior level is seen in parallel with activity at neuronal level throughout the S-R arc, i.e., from perception to action.
APPENDIX
Suppose four numbers V1,V2, V3, andV4 are given, representing neuronal activities on four distinct types of trials. Suppose the numbers are meaningful on an interval scale, i.e., the quadruplet can be subject to an arbitrary shift δ (representing zero-reference, for example) and a common scale factor k (representing unit of measurement, for example):
Equation 1AConstruct the pair-wise, squared differential D of these four numbers (in the main text, this refers to the differential activity (DA) or DA measure):
Equation 2Awhich, after an affine transform of theVi’s according to Equation A1, becomes:
Equation 3AThis is to say, D specifies the relative dispersion of the four numbers V1,V2, V3,V4 up to the scale factor. Writing outD explicitly:
Equation A4
or in vectorial notation:
Equation 5Awhere V = (V1,V2,V3,V4)Tis a four-dimensional vector (of the data space) and the matrix:
Equation 6Ais symmetric and thus have real eigenvalues. The eigenvalues λ can be found by setting:
Equation 7ASolving for λ yields:
Equation 8Aand:
Equation 9AThe eigenvector corresponding to λ4 is:
Equation 10Aand the eigenvectors corresponding to the degenerate eigenvalues in Equation A8 span a three-dimensional subspace, the orthogonal coordinate base of which can be chosen as:
Equation 11ANow,t1,t2,t3,t4form a set of new orthogonal basis for the four-dimensional data space in which Vi can be expressed. Writing outD in this new base (and noting that λ4 = 0):
Equation 12AIntroducing a three-dimensional vector (X,Y,Z), where the vectorial components are, respectively, projections of the four-dimensional vector V onto the mutually orthogonal directionst1,t2,t3:
Equation 13A
Equation 14A
Equation 15A
In this way:
Equation 16AClearly, under an affine transform (Equation A1) of theVi quadruplet, the three-dimensional vector becomes:
Equation 17AIf V is a random vector, i.e.,V1, V2,V3, V4 are independent random variables (assumed Gaussian) with identical mean and variance, then, X,Y,Z, according to Equations A13–A15, are also random variables with zero mean and a certain variance ς0; they are independent because of orthogonality oft1,t2,t3. Therefore, the value of their squared-sum D conforms to the χ2 distribution (Kendall and Stuart, 1977) with degree of freedom (df, n = 3):
Equation 18Awhere τ = D/ς0. The first two moments of the χ2 distribution can be evaluated:
Equation A19
where we have used the relation:
Equation 20ATherefore, the SD ςτ is:
Equation 21AThe mean (μD) and variance (ςD) of the random variable D are:
Equation 22AThey are linearly related:
Equation 23AThis proves that for random vector V = (V1, V2,V3,V4)T, the mean and variance of the constructed D measure obey a linear relationship, with a slope of 1.22 (for n = 3).
Introducing R = ✓D representing the length of the vector (X,Y,Z) (see Equation A16), we may construct spherical coordinates (x,y,z) that represent the intersection of (X,Y,Z) with the unit sphere X2 + Y2 +Z2 = 1:
Equation 24ATherefore, the vector (X,Y,Z) in the three-dimensional compressed space, and through Equations A13–A15, the vector V in the four-dimensional data space can be mapped onto a point on the unit sphere. We call this point the spherical locus of the quadruplet(V1,V2,V3,V4). When this quadruplet undergoes an affine transform (Equation A1), the three-dimensional vector (X,Y,Z) scales according to k: R′ = kR, yet the corresponding spherical locus remains unchanged. Therefore:
Proposition: The quadruplet (V1,V2,V3,V4) is uniquely specified by its spherical locus (x,y,z) up to an affine transform. There is a one-to-one correspondence (homeomorphism) of four numbers (on the interval scale) to a point on the sphere. Furthermore, spherical distance between any two points (x1,y1,z1) and(x2,y2,z2), as measured by the associated angle θ, defines the proximity (“similarity” or “resemblance”) between two such quadruplets:
Equation 25AThe special values of (V1,V2,V3,V4) that map onto a set of fundamental loci of the sphere are given in Table 1.
Pure spherical loci
The distance from a given spherical locus L = (x,y,z) to any of the above spherical landmark or fundamental locus L0 = (x0,y0,z0) is determined by the great circle that connects these two loci. The arc length between them is θR, where R is the radius (here assumed to be unity) and θ is the angleLOL0 (O is the origin):
Equation 26ATable 1 is consulted for the value (x0,y0,z0) of a particular fundamental locus. As (x,y,z) moves farther away from the reference locus, θ increases. If each of the 14 fundamental loci is considered to “possess” a neighborhood region that is defined by a circle (on the sphere) around the locus in question with certain cutoff value θ ≤ θ0, then the entire sphere contains 14 zones enclosing the set of fundamental loci, plus other regions that fall outside all such zones. The total area within the zones is:
Equation 27ABecause the total spherical area (for the unit sphere) is 4π, the total enclosed area has a fraction of
Equation 28AThe total amount of this “classifiable” area increases as θ0 increases. There is a critical value θ0= θc before the 24 zones start to invade each other. The critical value is half the distance between one of the primary locus and one of the conjunction locus:
Equation 29AThis gives cosθc = 0.888, and the total occupied area is A = 0.784 = 78.4%, whereas the rest 21.6% spherical area is deemed “unclassifiable.” The critical value is θc = 27.4°. On the other hand, if every point on the sphere is to belong to the closest one of the 14 locus (i.e., if every point is to be classified according to its closest fundamental locus), the radius of those zones will need to expand. It can be shown that as long as θ0 is increased to tan−1(✓3 −1) = 36.2°, all locations on the sphere will be possessed by one fundamental locus or another.
Footnotes
↵FNa Jean Requin died on June 21, 1996.
Correspondence should be addressed to Dr. Jun Zhang, Department of Psychology, 525 East University, University of Michigan, Ann Arbor, MI 48109.