Abstract
When we perform a skilled movement such as reaching for an object, we can make use of prior information, for example about the location of the object in space. This helps us to prepare the movement, and we gain improved accuracy and speed during movement execution. Here, we investigate how prior information affects the motor cortical representation of movements during preparation and execution. We trained two monkeys in a delayed reaching task and provided a varying degree of prior information about the final target location. We decoded movement direction from multiple single-unit activity recorded from M1 (primary motor cortex) in one monkey and from PMd (dorsal premotor cortex) in a second monkey. Our results demonstrate that motor cortical cells in both areas exhibit individual encoding characteristics that change dynamically in time and dependent on prior information. On the population level, the information about movement direction is at any point in time accurately represented in a neuronal ensemble of time-varying composition. We conclude that movement representation in the motor cortex is not a static one, but one in which neurons dynamically allocate their computational resources to meet the demands defined by the movement task and the context of the movement. Consequently, we find that the decoding accuracy decreases if the precise task time, or the previous information that was available to the monkey, were disregarded in the decoding process. An optimal strategy for the readout of movement parameters from motor cortex should therefore take into account time and contextual parameters.
Introduction
Neurons in motor cortex are well known to encode physical parameters of limb movements such as direction, force, extent, load, or posture (for review, see Hepp-Reymond, 1988; Johnson et al., 2001). In a series of experimental studies, it has been shown that encoding properties are sufficiently stable over time, such that arm movement trajectories could be accurately decoded from neural populations in single trials (Wessberg et al., 2000; Serruya et al., 2002; Taylor et al., 2002; Mehring et al., 2003). This opened the possibility to decode neural spike trains in brain–machine interfacing (BMI), as a means to control prosthetic devices (Carmena et al., 2003; Musallam et al., 2004; Hochberg et al., 2006; Santhanam et al., 2006; Moritz et al., 2008; Velliste et al., 2008). Indeed, real-time applications have, in principle, proven to work under experimentally controlled conditions. However, several recent studies provided evidence that neural encoding properties need not be stable under all circumstances. Instead, they can change dynamically with time, both within a trial and across repeated trials (Chestek et al., 2007; Churchland and Shenoy, 2007; Rokni et al., 2007). Such changes may be attributable to changes in contextual parameters. In parietal cortex, the influence of attention (Oristaglio et al., 2006), reward (Musallam et al., 2004), or movement cue (Gail and Andersen, 2006) on the neural encoding of movement goals is well established (for review, see Gottlieb, 2007). In motor cortex, there is evidence for similar influences on the encoding of movements. For instance, reward and reward expectation can modulate motor cortical activity related to eye movements (Roesch and Olson, 2004). Serial execution order of different movement components influences their neural representation (Carpenter et al., 1999; Hepp-Reymond et al., 1999; Lu and Ashe, 2005), as well as the presence or absence of specific movement cues (Hepp-Reymond et al., 1999). Moreover, motor cortical activity can be similar during actual execution and during the preparation of a movement (for review, see Riehle, 2005).
Here, we studied in detail the time-dependent encoding of movement direction in the monkey primary motor cortex (M1) and dorsal premotor cortex (PMd) during movement preparation and execution in a delayed hand-reaching task with a varying context. In different experimental conditions, two monkeys were confronted with different degrees of previous target information, presented at the beginning of the initial hold period. Previous studies showed that prior information modulates motor cortical activity during movement preparation (Georgopoulos et al., 1989; Riehle and Requin, 1989; Bastian et al., 2003; Roux et al., 2003; Cisek and Kalaska, 2005; Churchland et al., 2006) and during execution (Riehle and Requin, 1989; Hepp-Reymond et al., 1999). Our analyses, using single-trial Bayesian inference, show that the encoding of movement direction in M1 (monkey 1) and in PMd (monkey 2) is a dynamic process dependent on time and prior information. Nevertheless, on the population level, 50–100 neurons allowed an essentially error-free decoding of the visually presented cue and of the executed movement direction at any point in time during movement preparation and execution.
Materials and Methods
Experimental paradigm.
Two monkeys (Macaca mulatta), one male (monkey 1) and one female (monkey 2), were trained to perform a delayed arm-reaching task. The animals were cared for according to European and French government regulations. They were seated in front of a vertical panel carrying seven touch-sensitive buttons with light-emitting diodes (LEDs)—one in the center and six on a circle of 10 cm radius around it (see Fig. 1). When the center LED was lit, the monkey had to place its right hand on it to initiate the trial [trial start (TS)]. After a delay of 500 ms, one, two, or three adjacent target LEDs lit up in green [preparatory signal (PS)], providing the monkey with more or less information about the direction of the upcoming movement (see Fig. 1A): (1) in the one-target condition, one of the six target LEDs lit up; (2) in the two-target condition, one of three possible pairs of adjacent target LEDs lit up (targets 1 + 2, 3 + 4, or 5 + 6); (3) in the three-target condition, one of two possible triplets of adjacent target LEDs lit up (targets 6 + 1 + 2 or 3 + 4 + 5). After a fixed preparatory period of one second, one of the green LEDs turned red [response signal (RS)], signaling the monkey to release the center LED and to reach out and touch the red target signal. Thus, each condition comprised six types of trials (i.e., six possible combinations of the preparatory and the response signal). Correct and fast performance was rewarded by fruit juice. The three behavioral conditions of prior information (i.e., one, two, or three targets as preparatory signal) were presented as separate blocks of ∼100 trials each, within which each trial type was presented at random with equal probability (median of 19 trials per final target direction with a SD of 6.1 in monkey 1 and 13 trials per final target direction with a SD of 4.2 in monkey 2). On each recording day, the order of blocks was chosen randomly. Mean reaction times were 161, 195, and 204 ms for monkey 1, and 234, 236, and 237 ms for monkey 2, for the one-, two-, and three-target conditions, respectively. Mean movement duration was 199, 202, and 209 ms for monkey 1, and 181, 177, and 184 ms for monkey 2 (for additional details on the relationship of reaction time and prior information, see Bastian et al., 2003). To simplify terminology, the 1 s period between PS and RS was termed “preparatory period” and a 300 ms period from RS to ∼100 ms before movement end was termed “execution period.”
Electrophysiological recordings.
After the monkey had successfully learned the task (>85% correct performance), a cylindrical recording chamber (inner diameter, 15 mm) was implanted over the contralateral M1 and the PMd between central sulcus (CS), the posterior bank of the arcuate sulcus, and the precentral dimple, under aseptic conditions and general halothane anesthesia (<2.5% in air). In monkey 1, the recording chamber was positioned close to the CS (see Fig. 1B) and electrodes were positioned in M1; in monkey 2, the chamber position was more frontal (see Fig. 1C) mainly over PMd. A stainless-steel T-bar was cemented to the skull to fixate the animal's head during recording sessions. A multielectrode microdrive (Reitböck system; Thomas Recording) was used to transdurally insert up to seven quartz insulated platinum–tungsten electrodes (outer diameter, 80 μm; impedance, 2–5 MΩ at 1 kHz). Electrodes were spaced ≈300 μm apart from each other. The use of a window discriminator yielded the spike times of up to two neurons per electrode. The neuronal signals along with behavioral events (trial start and end, target information, reaction and movement times, reward, errors) were stored for off-line analysis at a time resolution of 1 ms. Here, we used only correct trials for analysis.
Estimation of neuronal firing rates.
In most cases (see Figs. 2⇓⇓⇓⇓–7A), spike trains were aligned to the occurrence of PS (for an example, see Fig. 2A). Neuronal firing rate profiles were estimated from single-trial spike trains (Nawrot et al., 1999) by convolution with a fixed Gaussian kernel of a standard width of 65 ms and area normalized to 1 (see example in Fig. 2B). The exact width of the kernel was not critical to our results. Kernels of widths in the range of 20–100 ms lead to the same general conclusions.
Measures for directional tuning.
We considered movements into six different movement directions d that were executed after RS. They were specified by the six uniformly arranged unit vectors e⃗d. For a sextet of neuronal firing rate profiles rd(t), one for each direction d, a fundamental quantity is given by the weighted vectorial sum as follows:
At each point in time, the resulting vector a⃗(t) represents the instantaneous directional tuning. Its length a(t) describes the tuning amplitude, or directional modulation. Its direction is specified by the unit vector a⃗(t)/a(t) or, alternatively, by the angle ϕ(t) with respect to the upward direction. Let 〈·〉 denote trial averages. The trial-averaged tuning vector A⃗(t) is the vector mean computed from the direction-specific trial-averaged firing rates Rd(t)≡〈rd(t)〉 as follows:
Let A(t) denote the length of the trial-averaged tuning vector A⃗(t). The variance of tuning can be predicted from the variances of the rates rd(t), provided that the latter are uncorrelated across directions. Two biologically relevant aspects of tuning variance are used to assess the reliability of directional tuning:
-
Let
denote the scalar variance of the tuning vector (i.e., the expected squared deviation of the tuning vector from its mean). The z score relating the length of the tuning vector to its scalar variance
is used as a measure for the reliability of the directional tuning. -
Circular statistics (Batschelet, 1981) allowed us to evaluate the preferred direction without regard of the tuning amplitude. Let
denote the mean preferred direction; A⊙(t), its length; and Φ(t), its angle with respect to the upward direction. The fact that
justifies the use of
as a measure for the angular variance. Its square root is called angular deviation, measured in radians. A uniform distribution of angles yields A⊙(t)=0 and an angular deviation of
For the preparatory period, these analyses were repeated with regard to the tuning strength of the preparatory stimulus instead of the movement executed after RS. Because we did not consider any error trials, this analysis made no difference for the one-target condition. In the two- and three-target conditions, the number of unit vectors reduced from six to three and two likewise. The results of these analyses were qualitatively the same as the above analyses with regard to the tuning strength of the executed movement. They are included in the supplemental material (available at www.jneurosci.org).
Bayesian inference of movement direction.
We wanted to know how accurately the movement direction executed after RS is represented in single-trial firing rates throughout the trial. To measure this, we used Bayesian inference from nonparametric density estimations. This method of Bayesian decoding assumes the perspective of an “ideal observer” (for review, see Pouget et al., 2000; Ma et al., 2006). It yields an estimate of the information content of neuronal activity without specifying the encoding strategy that is used by the brain. Our approach makes two constraining assumptions: (1) Firing rate is a sufficient descriptor of neuronal activity (i.e., the precise timing of action potentials is not considered to represent information), and (2) firing rates are assumed to be uncorrelated across neurons. The latter assumption enabled us to analyze pseudopopulations of neurons, most of them recorded in different sessions, but during trials in the same experimental condition.
In detail, this method constitutes the following steps: Trial-averaged tuning curves were replaced by probabilistic tuning profiles in terms of the following conditional probability:
Such probabilities are conveniently represented by appropriately normalized probability density functions. Estimates of these were obtained by kernel convolution with reflection to account for the nonnegativity of rates (Silverman, 1986). To assess the suitability of such a tuning model for the decoding of neuronal activity, we again used conditional probabilities, but now with the roles of r and d reversed:
measures the “plausibility” of a movement in direction d, given a specific observation of neuronal firing rate r. The Bayes rule was used to compute plausibilities from tuning profiles (Johannesma, 1981; Gelman et al., 1995; Pouget et al., 2000) as follows:
For any particular single-trial rate profile rd(t) recorded before and during a movement to direction d, the corresponding plausibility profile can be read out. This leads to six distinct probability profiles as follows:
measuring the plausibility of direction d′ if the rate profile rd(t) was observed. The expectation over 20 randomly chosen trials (in very few cases with <20 trials, some were used twice) for each movement direction with d′ = d gives the average probability
for the correct decoding at time t of the direction d of the actual movement that was performed. Note that the direction of the actual movement was in all cases identical with the direction indicated by the target cue presented at RS since we considered only correct trials. The training trials used to estimate these probabilities from experimental data did not include the respective test trial for which the plausibility profiles were compiled (cross-validation, leave-one-out) (Efron and Tibshirani, 1993). This provided us with an efficient means to assess the expected single-trial readout performance for a particular neuron; we refer to this measure as the decoding probability.
To compute whether the decoding probability of a neuron lies significantly above chance level, we compared the decoding probability to a chance distribution of decoding probabilities computed from a flat tuning curve. The chance distribution was obtained numerically by computing 100 times the decoding probability from single-trial rates of the same neuron but the rates were randomly sorted into six groups. To limit computing time, decoding probabilities at any time during the preparatory period were compared with one chance distribution computed from the mean firing rates of the neurons during the preparatory period. Likewise, decoding probabilities any time during the execution period were compared with one chance distribution from the mean rates during the execution period.
Next, we considered a neuronal population comprising n neurons. The joint decoding of its activity given by the rates r1, r2, …, rn should be based on the joint tuning curves as follows:
The assumption of independent encoding allows us to write the following:
Based on these joint probabilities, we can deal with neuronal populations in very much the same way as we did with single neurons.
Furthermore, we computed decoding probabilities based on different combinations of behavioral conditions (one, two, and three targets) used for training and test trials: (1) “context known”: same condition used for training and test trials; (2) “context unknown”: all three available conditions used for training trials; (3) “generalization”: training trials taken from a single condition different from the test trials condition.
In formal terms, using the Bayes probability for multiple variables, Equation 8 in our computation has to be replaced by the following:
In our case, all conditions could be considered equally likely, such that the equation simplifies to the following:
This reflects the context known case and is exactly what was used to compute the decoding probabilities in Figures 3⇓⇓⇓–7A. The other two cases are reflected by the following two equations:
where all available training data are used to approximate the probability of a direction when the context is unknown, for the context unknown case, and
where training data from another conditionB are used to approximate the probability of a direction with a different context (A), for the generalization case.
In the procedure described above, throughout the trial we inferred the movement direction that was instructed with RS and executed shortly after (only correct trials). For the two- and three-target conditions, target information was incomplete during the 1 s preparatory period. In the three-target (two-target) condition, the monkey could not know which of the three (two) possible targets would light up with the RS. Thus, before RS we expect that the maximally possible decoding probability for the final target is Pc = 1/3 (Pc = 1/2), given that there was no bias for any specific target in our experimental design.
Inference of the preparatory stimulus.
Alternatively, we computed the neuronal representation of the stimulus (rather than that of the final movement direction). Here, the probability of observing stimulus s′ given the firing rates rs was calculated using the same formal approach that led to Equations 9 and 12. In the two- and three-target conditions, we have three or two preparatory stimuli (PSs), and thus the decoding is reduced to a three- or two-class problem, respectively. The maximally possible decoding probability for the stimulus now is Pc = 1 for all stimuli, regardless of the condition. These results were qualitatively similar to those for the decoding of the final movement. They are included in the supplemental material (available at www.jneurosci.org).
Inference of task time.
We estimated the task time (in the range from 300 ms before PS to 300 ms after RS) from single-trial firing rates computed as described above. From a test data vector of single-trial firing rates (r1, …, rn) of a population of n neurons, we computed for each neuron i and at each point t in time the squared error SE between the firing rate of the neuron averaged across all trials and across all directions R(t) and the single trial test firing rate ri as follows:
The point in time for which the integrated squared error was minimal was used as the estimated task time T
The inaccuracy TA of time inference was quantified as the absolute difference between estimated task time T and the true time ttrue from which test data rn were taken. Average accuracy in task time inference at each ttrue was computed across all single trials by leave-one-out cross-validation and then taking the mean across all test trials. Chance level was computed from the average inaccuracy achieved with random values assigned to T (chosen randomly from the 1600 ms time span from 300 ms before PS until 300 ms after RS).
Results
Neuronal tuning is a function of time
In each of two monkeys, we analyzed the activity of 112 and 110 neurons, respectively, recorded under three different experimental conditions (Fig. 1). Typically, single neurons exhibited a marked temporal modulation of their spiking activity after the PS, which provided more or less prior information about the final movement target presented with the RS. Figure 2 shows a typical example of single-neuron activity under these conditions. As for most neurons, the trial-averaged rate profile (Fig. 2B, black curves) showed a clear dependence on movement direction, both during movement execution (interval from RS to RS plus 300 ms) and during the preparatory period (1 s from PS to RS) preceding the movement. Note, however, that the single trial spike responses (Fig. 2A) and rate profiles (Fig. 2B, gray curves) were highly variable across trials.
A, Experimental paradigm. Monkeys performed arm movements to one of six targets arranged on a circle around the central resting position. The trial started (TS) after the monkey had placed its hand on the central button. The PS was given one-half of a second later, by randomly choosing one peripheral button to be lit in green (top). Alternatively, two or three adjacent targets were illuminated in green (middle, bottom). The PS was followed by a 1 s preparatory period during which the monkey was not allowed to move its hand. The RS was given by turning one randomly selected green button into red. Correct and fast touching of the target was then rewarded after a short delay. MO and movement end (ME) are indicated by horizontal bars (mean ± SD) below the time axis. B, Location of recording chamber (red hatched circle) in monkey 1. PS, Principal sulcus; AS, arcuate sulcus; PrS, precentral sulcus; CS, central sulcus. C, As in B for monkey 2.
Single-trial responses of one neuron in the complete information condition. A, Raster plots of spiking activity during individual trials. Each panel corresponds to the direction indicated at the center. Each dot represents an action potential (spike), and each row represents a trial. B, Single-trial firing rates (gray) estimated from the spike trains in A. Trial-averaged rate profiles are shown in black.
The time-dependent directional tuning of the mean firing rate of a neuron was visualized with the tuning vector (i.e., the vectorial sum of the trial-averaged firing rates from each movement direction executed after RS) (see Materials and Methods). During the time course of the trial, the resulting vector plot expresses the temporal evolution of both, the amplitude of tuning (vector length), and the preferred direction (vector direction) (for examples, see Fig. 3A, top; B,C, top panels). Before PS, the monkey had no information about the movement target, and thus, neuronal activity could not have been tuned to target direction. This resulted in very small residual amplitudes and random directions of the tuning vector, reflecting the estimation bias attributable to limited sample size (Fig. 3A–C, top panels). After PS, the vector amplitude increased and the vector direction stabilized as the tuning of the neuron evolved with time.
Dynamic tuning properties of single neurons in motor cortex. A, Same neuron as in Figure 2 in the complete information condition. Top to bottom: (1) Time-resolved tuning vector A, sampled in 40 ms windows, pointing toward the instantaneously preferred direction; the vector length (circular mean) measures the average amplitude of tuning; (2) the SD of the vector length, σA, measures amplitude variability across 50 random single-trial sextets; (3) Z-score measures tuning strength as the ratio of the vector length, |A|, and σA; (4) angular deviation sA measures uncertainty in preferred direction across 50 single-trial sextets in degrees; the dotted line represents the chance level (sA ≈ 81°); (5) decoding probability Pc for an arbitrary single-trial (solid line) and trial-averaged firing rate profile (dash-dotted line); the dotted line again represents the chance level (Pc = 1/6). B, Tuning vectors (top), decoding probability Pc, and mean firing rate (bottom) for three different neurons in the complete information condition. C, As in B, but for one single neuron in all three prior information conditions, as indicated. The line styles in B and C are as in A. The calibration of vector length |A| indicated in A (top) is fixed for the entire figure.
As a consequence of trial averaging, the vector plot is much less susceptible to the high trial-by-trial variability of the neuronal activity. To account for the variability across trials in a statistical analysis of tuning based on single trials, we compared three different measures (see Materials and Methods), each of which provides us with a time-dependent estimate of the tuning strength of the neuron, which we refer to as the tuning profile of the neuron.
The first measure assesses the reliability of the tuning vector by computing at each point in time its z score zA(t) as the quotient of the mean amplitude and its SD σA(t) across 50 random sextets (one for each of six directions) (see Materials and Methods) of single trial rate profiles. Values >1 indicate strong and reliable tuning (mean amplitude larger than trial-by-trial SD). The example neuron in Figure 3A shows values clearly larger than 1 during most of the preparatory period (zA,max = 3.0) and again during movement execution (zA,max = 1.6). The average z scores for all neurons in all three conditions are given in Table 1.
Different tuning measures averaged across all neurons during the preparatory period from PS + 150 ms to RS − 150 ms, and during the execution period from RS to RS + 300 ms
Note that our results for all tuning measures during the preparatory period remained qualitatively the same when computed with respect to the stimulus presented with PS instead of the movement executed after RS (see supplemental material, available at www.jneurosci.org).
The second measure assesses the angular deviation sA(t) of the tuning vector computed across the preferred directions of 50 sample vectors, constructed once more from randomly chosen single-trial sextets. This measure translates the trial-by-trial variability of the firing rate into an angular uncertainty of the directional tuning. The neuron selected for Figure 3A, analyzed in the complete information condition, exhibited an angular deviation <30° during most of the preparatory period. This deviation was relatively small compared with the 60° angle between two adjacent movement targets, signaling strong tuning. The average angular deviations are provided in Table 1.
As a third measure, we computed the decoding probability (i.e., the Bayesian probability to correctly decode the direction of the movement executed after RS from the single-trial firing rates) (see Materials and Methods). Our decoding model was repetitively trained with all trials from one out of the three behavioral conditions, except for one test trial (cross-validation) (see Materials and Methods). In Figure 3A, the decoding probability averaged across all test trials Pc(t) rose shortly after PS from chance level (1/6, one in six directions) to a higher level at around 0.3, which persisted during most of the preparatory period. It dropped to values <0.2 around RS, after which it rose again to peak (Pc of 0.26) during the actual movement. The average Pc scores are given in Table 1.
Comparing panels 3–5 in Figure 3A, one observes that all three measures of tuning, z score zA, angular deviation sA, and decoding probability Pc, measured the tuning profile of this neuron in a consistent manner: during the control period before PS, the absence of information about the forthcoming movement target was correctly reflected by a z score close to zero and by an angular deviation and decoding probability at chance level. During both preparatory period and execution period, the three measures were strongly correlated, as indicated in Table 2. In the following analyses, we focused on Pc to study in detail how tuning strength and single trial decoding of movement direction depended on time and prior information.
Correlation between the three tuning measures during the time from PS to RS + 300 ms
Analysis of the decoding probability Pc(t) in all single neurons recorded from two monkeys showed that the profile of each neuron was unique and highly dynamic in time (for five examples, see Fig. 3A,B,C, top panel). The broad range of different tuning profiles found in our data is exemplified in the three examples shown in Figure 3B: a neuron that exhibited tuning only during the preparatory period (top), a neuron that exhibited tuning only during the execution period (middle), and a neuron that exhibited tuning during both phases (bottom).
Neuronal tuning depends on prior information
Thus far, we considered only one behavioral condition in which the final target was already presented by the PS. In this case, complete information on target direction was available to the monkey throughout the preparatory period. In the alternative cases, either two or three adjacent targets were presented at PS, of which only one was chosen randomly to be presented as RS (Fig. 1). In these cases, therefore, during the preparatory period prior information about final target direction was incomplete.
We found that the tuning profiles of neurons were not only highly dynamic with respect to time, but also depended strongly on the amount of prior information: For example, the neuron depicted in Figure 3C exhibited clear tuning during the preparatory period, but no tuning during movement execution in the one-target condition (top panel). In both conditions of incomplete previous target information (bottom two panels), however, the neuron exhibited tuning after RS: there, the decoding probability Pc rose to 0.23 around movement onset (MO) in the two-target condition and to 0.27 in the three-target condition. During the preparatory period, however, this relationship was reversed: Pc decreased from a broad maximum of 0.33 in the middle of the preparatory period in the one-target condition to somewhat lower maxima of 0.29 and 0.23 early in the preparatory period in the two- and three-target conditions, respectively. This behavior suggests that this neuron dynamically allocated its processing power according to the availability of directional information.
Such dynamic behavior was also reflected in the population average of Pc across all 112 (monkey 1) and 110 (monkey 2) single neurons. Figure 4A shows that, starting 150 ms after PS and throughout the remaining preparatory period, the average decoding probability reflected the amount of prior information: Pc was highest in the complete information condition (black curve) and lowest in the three-target condition (light gray). After RS, the condition of complete prior information differed from the conditions of incomplete prior information. For complete prior information, the maximum in Pc was shallower for both monkeys and was reached earlier in monkey 1. For incomplete prior information, the rise in Pc was steep and reached a higher maximum value in both monkeys (p < 0.001, Wilcoxon's rank sum test). Again, the alternative measures of tuning strength, z score, and angular deviation, confirmed this behavior in both monkeys (Table 1).
Dynamic representation of movement direction depends on prior information about the movement target. A, Time-resolved decoding probability Pc(t), averaged across 112 neurons in monkey 1 (M1; left) and 110 neurons in monkey 2 (PMd; right). The gray level indicates the amount of prior information available to the monkey after the preparatory signal PS: one target (black), two targets (dark gray), and three targets (light gray). B, Population average of the single neuron rate profiles for monkey 1 (left) and monkey 2 (right) in all three conditions of prior information (gray scaled as in A). RS indicates the appearance of the reaction signal.
Previous findings suggested a net flow of movement relevant information from premotor areas to M1 (Shen and Alexander, 1997b; Kakei et al., 2001, 2003; Cisek, 2006). Consistent with this hypothesis, we found that the single-trial decoding probability Pc in Figure 4A reached its initial plateau earlier (150, 160, 180 ms after PS for one, two, and three targets) in PMd (monkey 2) than in M1 (250 ms, all conditions, monkey 1). This is also reflected in an earlier increase of the average firing rates in monkey 2.
Relationship between neuronal tuning and firing rate
We found that the neuronal tuning and the average firing rates across directions followed quite different behaviors in time. First, the time-resolved decoding probability Pc(t) (Fig. 4A) and the time-resolved average firing rates (Fig. 4B) exhibited different temporal profiles: the rate peaked earlier, ∼125–160 ms after PS, and then dropped to the same level in all three information conditions. In contrast, Pc peaked ∼150–250 ms after PS and then stayed at a different level for each of the three prior information conditions. Also, in monkey 1, in the complete information condition, Pc rose to a second peak around RS, whereas the rate remained constant.
Second, the temporal dynamics of rate profiles and decoding probabilities showed remarkable variations across individual neurons and across prior information conditions (see examples in Fig. 3). The neuron in the bottom panel of Figure 3B, for example, strongly increased its tuning during movement preparation, whereas its firing rate averaged across directions remained relatively constant. The neuron depicted in Figure 3C behaved differently: During the preparatory period of the complete information condition, firing rate and decoding probability increased and decreased in parallel, whereas in the two- and three-target conditions tuning became weaker or even dropped to chance level, whereas the rate remained as in the complete information condition. During the movement period of the two- and three-target conditions, the relationship of rate and tuning differed again: now, tuning rose and peaked, whereas the rate decreased or remained at a low level.
Third, the occurrence of high decoding probability was only weakly correlated with high firing rates. In supplemental Figure 1 (available at www.jneurosci.org as supplemental material), we show scatter diagrams of the relationship between firing rate and Pc at two points in time (150 ms after PS and 150 ms after RS). High values for Pc occurred for a broad range of firing rates. Vice versa, high average firing rates resulted in a wide range of decoding probabilities. Across all recorded neurons, firing rates and Pc were weakly, but significantly, correlated with values between 0.15 and 0.45, depending on time, condition, and monkey (supplemental Fig. 1, available at www.jneurosci.org as supplemental material).
There are several possible explanations for these observations. For example, an increase in decoding probability could result from an increase of the firing rate for the preferred direction and a decreased rate for the antipreferred direction, leading to no change in the average firing rate. Or a more dispersed distribution of firing rates across trials (i.e., a larger trial-by-trial variability) would lead to more uncertainty of the previous probabilities and thus to a reduced decoding probability even for a fixed average firing rate. Likewise, less variability along with reduced firing rates could lead to increased decoding probabilities with decreasing average rates (see Discussion).
Distribution of tuning strength and tuning duration
To quantify the contribution of individual neurons to the tuning properties of the neuronal population, we examined the distribution of tuning strengths and tuning durations in the recorded neural populations. The time-resolved distribution of decoding probabilities Pc shown in Figure 5 reveals that at any point in time after PS only a fraction of neurons exhibited values of Pc above chance level. The percentage of neurons exhibiting a significant Pc (p < 0.001) (see Materials and Methods) at any given point in time is indicated by the black curves in Figure 5. During the preparatory period, this fraction increased monotonically with the amount of prior information: one-target condition, 50% (top panel); two-target condition, 40% (middle panel); three-target condition, 20% (bottom panel). For monkey 2, this fraction rose from ∼25, 20, and 15% (one-, two- and three-target conditions, respectively) at the start of the preparatory period to approximate levels of 50, 40, and 30% at its end. After RS (i.e., during movement execution), we found for both monkeys that the fraction of neurons with a significant decoding probability was higher in the two- and three-target conditions than in the condition of complete prior information. At the same time, also the absolute values of Pc were higher for two or three presented targets than for one target only.
Time-resolved distribution and proportion of significant single-trial decoding probabilities for the population of 112 neurons in monkey 1. The color code indicates for each point in time t (abscissa) the probability distribution of decoding probabilities Pc(t) (left ordinate) as computed for all 112 neurons. The hot colors indicate a peak in the distribution, and the cool colors indicate a low probability to observe a given value of Pc; white color reflects probabilities <0.01. The black curve indicates the percentage of significantly (p < 0.001) tuned neurons (right ordinate) as a function of time. Before PS the number of significantly tuned neurons is zero, and the distribution of Pc values is centered at the chance level (1/6). During the preparatory period, the fraction of significantly tuned neurons reaches the highest level for the one-target condition (top) and the lowest for the three-target condition (bottom). The distribution of Pc is skewed, and most neurons exhibit only small decoding probabilities. During movement execution (after PS), the fraction of significantly tuned neurons rises and peaks at ∼60% in the complete information condition and 70% in the two- and three-target conditions. At the same time, more neurons show stronger tuning in the conditions in which directional information was incomplete before RS than in the one-target condition.
We saw that individual neurons exhibited a characteristic temporal modulation of their tuning strength (see temporal profiles in Fig. 3). Therefore, the contribution of each individual neuron to the population average (Figs. 4, 5) changed over time. To quantify the single neuron contribution statistically, we counted for each cell the uninterrupted episodes of significant tuning (p < 0.001, see Materials and Methods) and measured their durations. For this analysis, we pooled all neurons from both monkeys. We found that, in any of the three information conditions, <10% of all neurons never reached significance throughout the task. Generally, the numbers of significant episodes per neuron (Fig. 6A) were similar for the three behavioral conditions: the majority of the neurons exhibited one or two episodes of significant tuning (55, 56, and 53% of all neurons in the one-, two-, and three-target condition, respectively). Their duration of uninterrupted episodes varied with the behavioral condition (Fig. 6B): most often it was quite short (mean duration <300 ms in 63, 73, or 86% of all neurons in the one-, two-, or three-target condition, respectively). For complete target information, a considerable fraction of 12% of all cells exhibited long tuning episodes, lasting for at least 900 ms. This means that these neurons stayed tuned during a long fraction of the preparatory period, and also during most of the movement execution phase. One such example is shown in Figure 3B (bottom panel). In the two- and three-target conditions, in contrast, in which movement preparation could only be completed after RS, the fraction of neurons exhibiting long tuning episodes (>900 ms) dropped to 7 and 2%, respectively. Together, the distribution of tuning strength and tuning duration in the recorded neurons shows that (1) at any time only a few neurons are significantly tuned, and (2) individual neurons are tuned at different instances in time. Thus, directional information is distributed across a subset of neurons and the composition of this neuronal ensemble varies over time.
Distribution of number and duration of episodes with significant tuning across all neurons from both monkeys (N = 222 for each prior information condition). A, Histogram of the number of episodes per neuron in which the decoding probability Pc was above chance (p < 0.001) (see text for additional explanation). B, Histogram of the mean duration of the episodes in A. Bar gray levels represent the amount of prior information as indicated.
Effect of prior information on population coding
How accurately can we infer the direction of movement during movement preparation and execution from single-trial activity of a population of neurons? To measure this, we constructed a pseudopopulation by randomly selecting N neurons from all recorded single neurons in one monkey. For a given degree of prior information, we then randomly selected one trial per neuron, each corresponding to the same movement direction. From this N-tuple of single-trial rate profiles, we estimated the average time-resolved decoding probability Pc(t). This procedure was then repeated with different nonoverlapping subsets of N neurons to obtain the average decoding probability for a given population size N (see Materials and Methods). By varying the number N, we measured the dependence of decoding probability on population size; results are shown in Figure 7.
Decoding movement direction from a neuronal population (monkey 1). A, Increase in decoding probability with population size (computed every 50 ms). Decoding probabilities are shown for a prediction using 1, 5, 10, 20, 50, and 100 neurons, respectively. Each panel shows the results for one prior information condition (from top to bottom: complete, two targets, three targets). Neurons were chosen randomly from the pool of neurons characterized by the distributions in Figure 5. Each neuron contributed only once to a population of a given size. In each panel, the dotted line depicts the chance level of decoding, and the dash-dotted line reflects the amount of prior information given to the monkey. B, Decrease in decoding probability when the training trials came from a disjoint information condition (red curves). The black curves show the decoding probabilities obtained, when test and training trials were obtained from the same condition. Populations of 10 and 50 neurons size were compared. From top to bottom, test trials belonging to one-target, two-target, and three-target conditions. Data were aligned to MO. The dotted and dash-dotted lines are as in A. C, Comparison of decoding probabilities when the training trials were comprised of all three experimental conditions (green curves). Otherwise, as in B. See text for additional explanation. Note that the decoding probabilities obtained for larger population sizes were based on averages over fewer pseudopopulations, which resulted in noisier curves.
As expected, Pc increased monotonically with neuron population size (Fig. 7A). During the preparatory period, Pc is naturally bounded by the amount of prior information available to the monkey. Thus, in the three-target case, the final direction was predetermined with a probability of 1/3 (and 1/2 in the two-target case, respectively) as indicated by the dash-dotted lines in Figure 7. In all three prior information conditions, a population size of ∼50–100 randomly selected neurons was sufficient to reach this upper bound already shortly (<200 ms) after PS, the directional information remained at this level throughout the preparatory period. The average Pc during the preparatory period (beginning 150 ms after PS) for 100 neurons in the complete information condition was 0.90 (0.79) for monkey 1 (monkey 2). In the two- and three-target conditions, decoding probabilities of 0.49 (0.42) and 0.30 (0.29) were achieved. Shortly after the RS, when complete target information was available in all conditions, the decoding probabilities assumed values >0.98 for both monkeys in all conditions. Note the increased variations in the decoding probability for larger population sizes. This is because these curves were necessarily based on averages over fewer pseudopopulations, resulting in noisier curves. Note also that, in our analysis without error trials, Pc values near the upper bound in all three conditions (Fig. 7A) also indicate that the motor cortex accurately represented the instruction stimuli of all three conditions. Pc values for stimulus decoding here reached values close to unity also for the two- and three-target conditions (supplemental Fig. 3, available at www.jneurosci.org as supplemental material).
Taking account of the distribution of tuning strength and tuning duration (Figs. 5, 6), the decoding probability of the neuronal population is based on neurons of which only a fraction of ∼20–50% is significantly tuned at any point in time. Of these neurons, only a few showed a high level of tuning strength, whereas most neurons were weak encoders of movement direction [consistent with previous results (Wessberg et al., 2000; Mehring et al., 2003; Hatsopoulos et al., 2004)]. To illustrate the potential gain by selecting a small neural ensemble, we picked the 10 neurons that exhibited the highest time-averaged Pc during the preparatory period and found that their decoding probability was comparable with that of 50 randomly chosen neurons (supplemental Fig. 4, available at www.jneurosci.org as supplemental material).
For all decoding probabilities computed thus far, the trials comprising the training data were taken from the same prior information condition as the test trials (i.e., the context was assumed to be known). With regard to a more general approach, such decoding strategy requires that the condition of the test trial is known and training data for this condition are available. In practical applications (e.g., for brain–machine interfacing), this requirement may not always be satisfied. We therefore examined the decoding probability also for two alternative scenarios.
First, we investigated generalization. For this, we computed the decoding probability when the training trials were exclusively taken from a different experimental condition than the test trials. This reflects the case in which a decoding algorithm is forced to generalize from one particular experimental condition (training data) to a different condition (test data). Figure 7B shows that generalization (red curves) led to a considerably decreased decoding probability compared with the case of a known context (black curves), both during preparation and execution of movement.
Second, we computed the decoding probability when the training trials were pooled from all three prior information conditions (context unknown). This reflects the case in which the decoding algorithm was trained with data from multiple conditions, but the specific context of the test trial is unknown. We found that the decoding probability (Fig. 7C, green curve) was again lower (but less so than compared with the generalization scenario) (Fig. 7B) than in the case of a known context (black) in which training and test trials were taken from the same condition only. Thus, the threefold increase in the number of training trials obtained by pooling over conditions could not fully compensate for the increased variability in the training set. With a population size of 50 neurons, however, decoding probabilities were nearly as good as in the control case. Only for test trials belonging to the complete information condition, the decoding probability was still considerably reduced during the preparatory phase.
Note that, for the analyses in Figure 7, B and C, trials were aligned to MO rather than to the PS, because we wanted to compare Pc(t) during the movement period, and to better eliminate the latency variability across trials and information conditions (see Materials and Methods). For the case of a known context (black curves), the alignment to either MO (Fig. 7B,C) or PS (Fig. 7A) yielded approximately equal decoding probabilities.
The difference in performance of our decoding procedure for different training scenarios is summarized in Figure 8. It shows that both scenarios, generalization and, to a lesser degree, context unknown, consistently lead to decreased decoding probabilities compared with the standard context known scenario. However, for higher numbers of neurons, the performance for context unknown was almost as good as for context known, and much better than for generalization.
Relationship of decoding probability with population size and prior information conditions used to train the decoder. Black bars, Training data taken from the same condition as test data only (context known); green bars, training data combined from all three conditions (context unknown); red bars, training data taken from a condition different from the test data (generalization). Computation performed for the firing rates at movement onset.
Decoding task time and movement direction
Our data show that both single neuron activity and tuning strength typically change in time during a trial (see sample neurons in Figs. 2, 3). We therefore examined how well task time itself can be inferred from single-trial firing rates. Figure 9A shows that task time (i.e., the point in time within a given trial as defined by the time frame of the experiment) can be decoded from the firing rates of neuronal populations. The time accuracies reached levels of ∼20–100 ms for a population size of 100 neurons. Specifically, by finding the minimal squared difference between the population firing rates at the test trial and the mean time-resolved firing rates of each neuron (see Materials and Methods), the average accuracies reached a minimum of 55, 58, and 56 ms during the preparatory phase (for one-, two-, and three-target conditions), and 42, 25, and 28 ms on average during movement execution (one-, two-, and three-target conditions) in monkey 1 (Fig. 9A). Monkey 2 displays a notably weaker time encoding during the preparatory phase (112, 113, and 128 ms for one-, two-, and three-target conditions) and stronger time encoding in the execution phase (29, 23, and 17 ms for one-, two-, and three-target conditions). This characteristic matches the weaker direction encoding during movement preparation and stronger direction encoding during execution in monkey 2. Studying the time course of the accuracy of time coding, it turns out to be higher around behaviorally relevant time points (i.e., PS and RS) than before PS or in-between PS and RS.
Inference of task time (A) and decoding of movement direction based on inferred task time (B) in monkey 1. A, The accuracy of the inferred task time T increased with neuronal population size. Mean accuracy, computed in steps of 50 ms, is shown for population sizes of 5, 10, 20, 50, and 100 neurons. The dotted curve depicts the chance level, which is time dependent because larger errors can occur at the beginning and at the end of a trial. Before PS and after RS, chance level is >500 ms. Neurons were chosen randomly as described in Figure 7A. B, Probabilities Pc(t) for decoding movement direction when the training data are chosen from a time point previously inferred from the test data (blue curves) are compared with the standard decoding approach when the correct time of the test data is assumed to be known (black curves) (compare Fig. 7A). Results for populations of 10 and 50 neurons are as indicated. The dotted and dash-dotted lines are as in Figure 7. Increased variation in the decoding probability across different points in time for larger population sizes is attributable to the reduced number of random populations that could be used. From top to bottom in A and B, One-, two-, and three-target conditions.
Our previous approach of decoding movement direction, as computed in Figures 3⇑⇑⇑–7, considered firing rates of the training data and of the test trial that were sampled at the same point in task time (i.e., the precise task time was assumed to be known and available to the decoding algorithm). In a second approach, we therefore examined whether one can in a first step decode task time and in a second step decode movement direction by selecting the training set on the basis of the decoded task time. Figure 9B shows the resulting decoding probabilities for populations of 10 and 50 neurons in all three prior information conditions (blue curves) compared with the standard approach in which time is known (black curves). As can be seen, movement direction can indeed be decoded without explicit knowledge of the task time from which the single-trial test data were taken, but at the price of a decreased accuracy. The notable decrease in Pc(t) when time was chosen with rather small imprecision (<100 ms) emphasizes the strong time-dependent characteristics of neuronal tuning properties.
Discussion
Dynamic encoding of movement direction
In both monkeys, single neuron tuning profiles could not easily be sorted into clear-cut categories with respect to number, starting time, and duration of significantly tuned episodes but rather described a continuum. Such heterogeneity of tuning properties has been previously described in the premotor and primary motor cortex (Shen and Alexander, 1997a,b; Cisek and Kalaska, 2005; Churchland and Shenoy, 2007). Here, we showed that this heterogeneity extends into the time domain. Statistically, most neurons exhibited one or two rather short (≤250 ms) uninterrupted episodes during which the preferred direction typically remained stable. In both monkeys, a fraction of ∼10% of all neurons showed an early increase in tuning strength shortly after PS, and then stayed tuned above chance level throughout the trial. This behavior is reminiscent of neurons involved in working memory, as described, for example, in prefrontal cortex (for review, see Romo and Salinas, 2003). In our experiments, however, the monkeys did not have to actively memorize the visual target cue, as it was present throughout the preparatory period. The tuning profiles of individual neurons were strongly modulated by the prior information about the upcoming movement. This was true also during movement execution (Fig. 3C), even though the RS and the executed movement were the same in all three conditions. This suggests that movement representation in the motor cortex is not organized according to “one movement–one code” and extends prior results on the influence of contextual information on movement parameter encoding (Georgopoulos et al., 1989; Riehle et al., 1997; Bastian et al., 1998, 2003; Hepp-Reymond et al., 1999; Roux et al., 2003; Cisek and Kalaska, 2005).
Combining both aspects of directional tuning, its modulation in time, and its dependence on prior information, leads us to the following interpretation of our data: During a delayed reaching task, computational demands vary with time. When the initial information is made available with PS, movement preparation starts and, in the case of complete target information, may be completed before RS. The initial peak of the average firing rate observed 150–200 ms after PS (Fig. 4) may, thus, reflect a high computational load on cells involved in movement preparation (“preprocessing” neurons) (Riehle and Requin, 1989; Riehle, 2005). With incomplete prior information, movement preparation could not be completed before the RS resolved the ambiguity in target location. In our interpretation, this additional information again created an increased computational demand, requiring additional neuronal resources for movement planning. This view is directly supported by the observation that the peak values of average firing rate and tuning strength after RS were higher for incomplete prior information conditions compared with the one-target case (Fig. 4) even though the same RS was presented in all three conditions.
The decoding probability increased monotonically with neuronal population size (Fig. 7; supplemental Fig. 3, available at www.jneurosci.org as supplemental material) and reached average probabilities Pc > 0.98 for the correct decoding of movement direction around movement onset for 100 neurons. Throughout the preparatory period, the neuronal population accurately reflected the stimulus (supplemental Fig. 3, available at www.jneurosci.org as supplemental material) and, likewise, the degree of prior information about the final movement target (Fig. 7).
From a computational perspective, our results suggest that, after PS, the motor cortex prepares a movement as early and accurately as possible, based on the available information.
Cisek and Kalaska (2005) raised the question whether, in case of ambiguous target information, the motor cortex prepares alternative movements simultaneously, or whether it adopts a “guess and switch” rule, in which only one movement is prepared (guess), which after final target specification is possibly switched. In their experiment, the monkey was confronted with two opposite (i.e., conflicting) movement targets, and their analysis supports the former model of simultaneous encoding. In our experiment, two or three adjacent targets increased the angle of possible movement direction. Thus, target information was not causing a conflict, but rather an inaccuracy. This opens the possibility for an alternative strategy, in which the motor cortex prepares the movement less precisely in the case of incomplete information.
Trial-by-trial variability and “optimal subspace hypothesis”
Trial-by-trial variability of single-neuron activity in the motor cortex changes with time and in relation to behavior (Oram et al., 2001; Nawrot et al., 2003, 2008; Churchland et al., 2006; Nawrot, 2009; Ponce-Alvarez et al., 2009). Previous analyses of our data showed that, in both monkeys, the initially high single-neuron variability dropped sharply shortly after PS, and again shortly after RS (Nawrot et al., 2000, 2001; Rotter et al., 2005). Therefore, the increase in Pc observed after PS and again after RS (Figs. 4, 5, 7) results from a combination of decreased variability and increased directional modulation of firing rates.
Churchland et al. (2006) formulated the optimal subspace hypothesis for movement preparation in the premotor and primary motor cortex. According to this hypothesis, each movement plan is represented by the activity of an adequate neuronal ensemble that occupies a subspace in the high-dimensional space of all possible activity patterns. After a target comes up, movement preparation leads to a gradually refined representation of the planned movement until the pattern of neuronal ensemble activity occupies the relevant subspace, resulting in the desired accuracy for a correct movement execution. The authors based their hypothesis on observations of neuronal variability matching with ours: variability decreased with the occurrence of a preparatory cue, and again with the GO cue. Additionally, we found in both monkeys an increase in variability during movement preparation from one target to two and three targets (supplemental Fig. 5, available at www.jneurosci.org as supplemental material) (Nawrot et al., 2009). Thus, our results support the optimal subspace hypothesis: in the two- and three-target condition, movement preparation could only be refined to a certain stage and, thus, occupied a subspace that is more extended than the optimal subspace associated with a unique movement direction.
Interpretational issues
There is strong evidence that processes associated with sensory-motor transformations are reflected in the activity of motor cortical neurons (Murata et al., 1997; Shen and Alexander, 1997a,b; Zhang et al., 1997; Kakei et al., 2003; Paz et al., 2003; Schwartz et al., 2004; Cisek and Kalaska, 2005). For example, it has been shown that, early in the preparatory period, neurons in PMd and M1 are preferentially tuned to the location of a visual target, and then gradually shift their tuning to movement direction (Shen and Alexander, 1997a,b; Zhang et al., 1997). Thus, the observed time-dependent involvement of neurons as discussed above may to some extent reflect a transformation from a stimulus representation in form of a visual target to a representation of motor output. It cannot be completely ruled out, however, that the recorded activities were, at least partially, influenced by additional factors, in particular the visual stimulus itself, eye movements, or attentional effects. However, we are convinced that our findings cannot be explained by these effects alone. The encoding of a visual stimulus could not explain why firing rates and tuning strengths are elevated at the end of the preparatory period, and why this late preparatory activity is correlated with reaction time (Riehle and Requin, 1993). Eye movements were reported to affect motor cortical activity (Boussaoud, 1995; Baker et al., 1999), but are unlikely to exert a major effect in our data since eye movements after PS were phasic and nonsystematic during the preparatory period (A. Riehle, unpublished observations). Similarly, attention is known to affect activity in premotor cortex (Boussaoud, 2001), but a dominant role of attention is contradicted by our finding of higher activities and less variability during the preparatory period in the complete information condition compared with both incomplete information conditions, in which attention would be expected to be higher.
Relevance for brain–machine interfaces
Our results suggest that, during movement preparation and execution, single-trial firing rates of 50–100 randomly chosen, or of ∼10 selected (supplemental Fig. 4, available at www.jneurosci.org as supplemental material) motor cortical neurons could allow an essentially error-free decoding of a two-dimensional movement direction. These numbers lie within the previously reported range of ∼10–20 neurons (Serruya et al., 2002; Taylor et al., 2002), and several tens to hundreds of neurons (Wessberg et al., 2000; Carmena et al., 2003; Musallam et al., 2004). In addition, we could show that it is feasible to jointly decode direction and task time albeit at a significant cost in performance for direction decoding [complementing the results of Achtman et al. (2007)]. Both results could be of practical relevance for future BMI applications in patients (Hochberg et al., 2006). Under realistic conditions, the time until the execution of an intended movement could be an important additional parameter to be estimated. Also, it will likely be advantageous for subjects that operate a brain-controlled prosthesis to make full use of contextual information, because it may help planning and executing a desired (prosthetic) limb movement. A first step to improve decoder performance could be its training in different context settings. In our off-line analyses, training with the pooled data from all three information conditions (context unknown) (Fig. 7B) allowed to partly compensate for the lack of knowledge about the actual condition. Models for decoding complex movements therefore might have to take into account various contextual parameters to successfully operate a BMI. Here, neuronal activity recorded from other brain areas may contribute to successfully decode context-related information (Shenoy et al., 2003; Musallam et al., 2004).
Footnotes
-
We gratefully acknowledge partial financial support from the German Federal Ministry of Education and Research [Bundesministerium für Bildung und Forschung–Deutsch-Israelische Projektkooperation, GoBio 0313891, and 01GQ0420 (Bernstein Center for Computational Neuroscience Freiburg) and 01GQ0413 (Bernstein Center for Computational Neuroscience Berlin)] and from the French Government (Action Concertée Incitative Cognitique “Variability and Invariance” and ANR-05-Neur-045-01). Additional funding was received from the Heidelberg Academy of Science and Humanities (M.P.N.). We thank Annette Bastian and Franck Grammont for performing the monkey experiments and Marc Martin for animal care. We thank the reviewers for constructive comments and their suggestion for the additional time decoding analysis that helped to improve this manuscript.
- Correspondence should be addressed to Jörn Rickert at the above address. rickert{at}biologie.uni-freiburg.de
