Abstract
We used large-scale computer simulations of eyelid conditioning to investigate how the cerebellum generates and makes use of temporal information. In the simulations the adaptive timing displayed by conditioned responses is mediated by two factors: (1) different sets of granule cells are active at different times during the conditioned stimulus (CS), and (2) responding is not only amplified at reinforced times but also suppressed at unreinforced times during the CS. These factors predict an unusual pattern of responding after partial removal of the cerebellar cortex that was confirmed using small, electrolytic lesions of cerebellar cortex. These results are consistent with timing mechanisms in the cerebellum that are similar to Pavlov's “inhibition of delay” hypothesis.
The timing displayed by Pavlovian eyelid responses represents an opportunity to examine how a brain system generates and makes use of temporal coding. Pavlovian eyelid conditioning involves repeated presentations of an initially neutral conditioned stimulus (CS)—often a tone—paired with a reinforcing unconditioned stimulus (US)—such as periorbital electrical stimulation. With this training the animal not only learns to close its eye in response to the tone but also learns to time or to delay these conditioned responses to achieve maximum eyelid closure when the US arrives (Schneiderman et al., 1962; Mauk and Ruiz, 1992). Previous studies have shown that lesions of the cerebellar cortex permanently abolish this adaptive timing, yielding conditioned responses with short and relatively fixed latencies (Perrett et al., 1993; Garcia and Mauk, 1998). Although these studies indicate that the cerebellar cortex is necessary for adaptive timing of conditioned response, the present studies attempt to address the mechanisms involved.
Pavlov (1927) first proposed in his “inhibition of delay” hypothesis that the timing of conditioned responses could be delayed appropriately via differential conditioning within each trial. In Pavlov's view, adaptive timing could be achieved with a brain mechanism able to discriminate the latter part of the tone, which is reinforced by the US, from early periods during the tone, which are not reinforced. Thus, the ability to respond at the right time would result from learning to suppress the response early in the tone as well as learning to respond later in the tone. Here we show that large-scale computer simulations of the cerebellum are able to learn adaptively timed responses in part via a variant of Pavlov's within-trial differential conditioning mechanism. Our simulations suggest that this mechanism is mediated by differential modification of synapses in the cerebellar cortex dependent on whether they are active early or late in the CS. The simulations reproduced previous empirical observations by producing short-latency responses when the cerebellar cortex was removed, but they also predicted a novel and unusual pattern of results with partial removal of the cerebellar cortex.
To test this prediction we examined the effects that small, electrolytic lesions of the cerebellar cortex have on the timing of conditioned responses. In the simulations the initial effect of a partial lesion is to produce conditioned eyelid responses with both short-latency and properly timed components. With additional postlesion training the well timed component persists, whereas the short-latency component diminishes or disappears completely. We present data from a group of rabbits in which we observed this pattern of effects after making electrolytic lesions of the anterior lobe of the cerebellar cortex. These results are compatible with a timing mechanism that actively learns to suppress the conditioned response early in the CS, consistent with Pavlov's inhibition of delay hypothesis.
MATERIALS AND METHODS
Computer simulations. The simulations were intended to capture the synaptic organization and physiology of the cerebellum (Eccles et al., 1967; Ito, 1984) and the way in which its inputs are engaged by eyelid conditioning (Fig. 1; Table 1). Converging evidence from a number of laboratories suggests that (1) the CS is conveyed to the cerebellum via the mossy fiber input (Steinmetz et al., 1985, 1988;Lewis et al., 1987), (2) the US is conveyed by the climbing fiber input (McCormick et al., 1985; Mauk et al., 1986), and (3) increases in the activity of cerebellar output cells in the anterior interpositus nucleus drive the expression of the conditioned eyelid response (McCormick and Thompson, 1984). Because of the straightforward manner in which CS and US map onto the afferent pathways to the cerebellum, eyelid conditioning can be relatively easily represented in our simulations. Adding a constant depolarizing pulse to the membrane potential of the climbing fiber simulates the presence of the US during acquisition trials (Sears and Steinmetz, 1991) (Fig. 1, CLIMBING FIBER INPUTS), whereas transiently altering the activity of the mossy fiber input as initially described by Aitkin and Boyd (1978)represents the CS (Fig. 1, MOSSY FIBER INPUTS). Increases in simulated nucleus cell activity can then be taken as a measure of the conditioned response (Fig. 1, NUCLEUS CELL OUTPUT).
The key assumption of our simulations is that two sets of synapses in the cerebellum can undergo changes in strength during motor learning. On the basis of evidence that supports this assumption (Robinson, 1976;Perrett et al., 1993; Raymond et al., 1996; Mauk, 1997), we have implemented a plasticity rule that specifies that the synapses that granule cells make with Purkinje cells (gr→Pkj) decrease in strength when active in the presence of a climbing fiber input and increase in strength when active in the absence of a climbing fiber input (Sakurai, 1987; Hirano, 1990; Salin et al., 1996). On the basis of indirect evidence (Perrett and Mauk, 1995; Medina and Mauk, 1999) that mossy fiber plasticity is controlled by Purkinje cell activity, the simulations also implement plasticity at mossy fiber synapses onto the cerebellar nucleus (mf→nuc) so that their strength decreases when they are active during periods of strong inhibition from Purkinje cells and increases during pauses in this inhibition. Although these synapses contain NMDA receptors (Cull-Candy et al., 1998) and there is one report of mossy fiber plasticity under rather nonphysiological stimulating conditions (Racine et al., 1986), the results did not depend on the precise plasticity implementation as long as conditioning strengthened mossy fiber synapses activated by the CS.
Animals. Data were obtained from nine male New Zealand albino rabbits (Oryctolagus cuniculus), weighing 2.5–3.0 kg each. The animals were individually housed and given food and waterad libitum. Treatment of the animals and surgical procedures were in accordance with an approved animal welfare protocol.
Surgical preparation. All animals were first prepared with a head bolt cemented to the skull and electrodes implanted in the anterior lobe of the cerebellar cortex (5.7 mm anterior, 4.9 mm left lateral, and 14.0 mm ventral to lambda). A large craniotomy was drilled to accommodate the electrode assembly and covered with bone wax. After placement, the electrode assembly and head bolt were secured to the skull with dental acrylic, and the skin was sutured. Two stainless steel stimulating electrodes were chronically implanted in the periorbital muscles rostral and caudal to the eye. Antibiotics, intravenous fluids, and analgesics were administered after surgery as needed, and animals were allowed ∼1 week to recover.
Conditioning procedures. The standard training session involved a Pavlovian conditioning delay protocol with a 500 msec interstimulus interval. Each training session consisted of 12 nine-trial blocks. Each block was comprised of eight paired presentations of the CS and US and one presentation of the CS only. The CS (a 1 kHz, 85 dB tone) was presented for 550 msec during CS-alone trials and coterminated with a 50 msec train of constant-current pulses (200 Hz; 1 msec pulse width; 1–2 mA) delivered to the periorbital electrodes during paired trials. Trials were separated by a fixed 30 sec intertrial interval. Animals were conditioned to an asymptotic rate of responding with at least 10 standard training sessions. Lesions were then made by passing 2.0 mA of direct current through one or more of the electrodes for ∼1 min. The timing of conditioned responses was assessed over six postlesion training sessions.
Stimulus presentation and data acquisition were controlled by a computer using custom software. Movement of the unrestrained eyelid was recorded by measuring the reflectance of an infrared light-emitting diode aimed at the eyelid. Voltage responses were determined to be linearly related to eyelid movement and were calibrated for each animal daily. Digitized responses (1 point per msec) were analyzed using custom software to determine onset and peak latencies.
Histology. After training, the location of the lesion was determined for each animal using standard histological procedures. Animals were killed with an overdose of sodium pentobarbital and perfused intracardially with 1.0 l of 10% formalin. The brains were removed and stored in 10% formalin for several days. Brains were embedded in an albumin–gelatin mixture. To assess the extent of the lesions, we sectioned the cerebellum parasagittally using a freezing microtome (80 μm sections). This plane of sectioning best illustrated the degree to which tissue damage involved each lobule of cerebellar cortex. Tissue was mounted, stained with cresyl violet, and counterstained with Prussian blue.
Data analysis. For rabbit data, peak response amplitude, onset latency, and peak latency were calculated by custom software. Digitized sweeps corresponded to the 200 msec before and 2300 msec after the CS onset. After being calibrated, peak amplitude was measured relative to an average of the 200 msec baseline collected before CS onset. To be counted as a conditioned response, onset latency had to follow CS onset, and the movement amplitude had to reach 0.3 mm before US onset during paired trials. This criterion was relaxed for CS-alone trials in which movements were counted as conditioned responses if they reached a 0.3 mm amplitude at any time after CS onset. Trials in which there was >0.3 mm of movement during the baseline were excluded from further analysis. Onset latency was determined by calculating the point at which the response reached criterion. For simulation data, the activity of the nucleus cells had to increase by 10 spikes/sec over background levels to reach criterion. This increase corresponds to 5% of the maximum rate of activity for the nucleus cells and thus is similar to the 0.3 mm criterion used for rabbit data (i.e., 0.3 mm is ∼5% of the maximum conditioned response amplitude).
RESULTS
Simulations learn appropriately timed conditioned responses
The architecture of the simulations was based on three fundamental properties (see Appendix for details). First, neurons were represented as single-compartment, leaky integrate-and-fire elements. This standard representation is based on the idea that the summation of inputs (excitatory and inhibitory) and leak currents determines a neuron's membrane potential. A spike occurs whenever the membrane potential exceeds a threshold level, and the refractive period is achieved by elevating the threshold immediately after a spike and allowing it to decay back to its original value. Second, layers of these integrate-and-fire neurons were interconnected in ways consistent with the known numeric ratios of cells, the geometry of the projections, and the divergence/convergence ratios of connections between cell types (Fig. 1) (Eccles et al., 1967; Ito, 1984). When known, the individual properties of each type of simulated neuron and its synapses were based on their cerebellar counterparts (Table 1) (Shinoda et al., 1987; Midtgaard, 1992; Barbour, 1993; Gould et al., 1993; Llano and Gerschenfeld, 1993; Puia et al., 1994; D'Angelo et al., 1995; Mouginot and Gahwiler, 1995; Virginio et al., 1995; Lang et al., 1996; Simpson et al., 1996; Vincent and Marty, 1996; Dieudonne, 1998; Kim et al., 1998). Third, on the basis of empirical evidence, two classes of synapses were modifiable according to specific, activity-dependent plasticity rules. The gr→Pkj synapses decreased in strength [long-term depression (LTD)] when active during a climbing fiber input to the Purkinje cell and increased in strength [long-term potentiation (LTP)] when active in the absence of a climbing fiber input (Sakurai, 1987; Hirano, 1990; Salin et al., 1996). As first suggested by Miles and Lisberger (1981), and supported by theoretical work (Medina and Mauk, 1999), the second site of plasticity involves LTD of mf→nuc synapses during strong inhibitory input from Purkinje cells and LTP during pauses in this strong inhibition. However, for the present results, the specific rule governing plasticity in the nucleus is not crucial as long as mossy fiber synapses by themselves are strong enough to permit responding during the expected decrease in Purkinje cell inhibition associated with a lesion of the cerebellar cortex. Interestingly, although evidence of LTP and LTD of these mossy fiber synapses is almost nonexistent in the literature (Racine et al., 1986), a recent report (Aizenman and Linden, 2000) has shown increases in the intrinsic excitability of nucleus cells after repeated mossy fiber stimulation. The results presented here would not be expected to differ if this form of nonsynaptic plasticity were implemented, assuming that conditioning can lead to an increase in the excitability of the nucleus cells.
Events occurring during the simulations were based on the well characterized relationship between the synaptic organization of the cerebellum (Eccles et al., 1967; Ito, 1984) and Pavlovian eyelid conditioning (Thompson, 1986; Thompson and Krupa, 1994; Mauk and Donegan, 1997) (Fig. 1). Previous studies have demonstrated that information about the CS and US is conveyed to the cerebellum via mossy fiber (Steinmetz et al., 1985, 1988; Lewis et al., 1987) and climbing fiber (McCormick et al., 1985; Mauk et al., 1986) inputs, respectively, and that output of the cerebellum via the interpositus nucleus is necessary for the expression of the conditioned responses (McCormick and Thompson, 1984). This correspondence between eyelid conditioning and cerebellar input–output pathways permits a relatively straightforward representation of Pavlovian conditioning in a simulation (Fig. 1). Presentation of the CS was simulated by altering the background activities of a small number of mossy fibers inputs (∼4%) according to published recording data (Aitkin and Boyd, 1978) (Fig. 1, MOSSY FIBER INPUTS). Similarly, applying a transient excitatory input to the climbing fiber input simulated the observed activation of these fibers in response to the presentation of the US (Sears and Steinmetz, 1991) (Fig. 1, CLIMBING FIBER INPUTS). Finally, based on evidence that stimulation of the interpositus nucleus elicits eyelid movements and that these neurons display increases in activity that precede and closely resemble the eyelid response (McCormick and Thompson, 1984), increases in simulated nucleus cell activity during the CS were taken as a measure of the amplitude and timing of conditioned eyelid responses (Fig. 1,NUCLEUS CELL OUTPUT).
Similar to what is observed during standard eyelid conditioning, when trained in this way the simulations gradually acquired appropriately timed conditioned responses over several hundred trials (see Figs. 2, 4a). As observed in chronic recordings during eyelid conditioning in rabbits, nucleus cell activity did not increase immediately after the onset of the CS; rather, the peak in activity occurred just before the presentation of the US regardless of the CS–US interval that was used during training (Fig.2). As described in detail below, these appropriately timed responses were produced in the simulation by a combination of plasticity in both the cerebellar cortex and cerebellar nucleus, which is consistent with results from previous studies on eyelid conditioning and vestibulo-ocular reflex adaptation (Robinson, 1976; Perrett et al., 1993; Raymond et al., 1996; Mauk, 1997). The simulations also produced extinction of conditioned responses when the CS was presented in the absence of the US (data not shown).
Although no unwarranted features were added to the simulations to ensure that they learn with appropriate timing, they necessarily contain inaccuracies and omissions. Thus, rather than a precise emulation of the cerebellum, the simulations can be considered the equivalent of a detailed and precisely specified working hypothesis of cerebellar mechanisms that contribute to eyelid conditioning. Like all hypotheses, the value of the simulation rests primarily with its ability to generate empirically testable predictions. Next we examine how the simulations accomplish response timing and then present results from a lesion experiment that tests a corresponding prediction.
Within-trial differential conditioning sharpens response timing
Response timing in the simulations arises from the following three features: (1) the activity of granule cells varies throughout the presentation of the CS such that the subsets of gr→Pkj synapses that are active at the beginning of the CS are somewhat different from those active at the end (Fig. 3), (2) conditioning results in increased inhibition of the nucleus cells by Purkinje cells during the early parts of the CS, and (3) conditioning also reduces the inhibitory action of the Purkinje cells during the latter parts of the CS. Figure 4,b and c, illustrates that as training proceeded, simulated Purkinje cells acquired the ability to decrease their activity late in the CS and to increase it early in the CS. The learned decrease was caused by the induction of LTD in gr→Pkj synapses active late in the CS, during the US-related climbing fiber input to the Purkinje cells. Similarly, the increase in Purkinje cell activity in the simulation was produced by the induction of LTP in gr→Pkj synapses active during the early parts of the CS, when the reinforcing climbing fiber input was not present. Interestingly, recent recording studies from Purkinje cells during classical conditioning have demonstrated the same unusual pattern of early-increase and late-decrease in activity shown by our simulated Purkinje cells (Hesslow and Ivarsson, 1994).
Because Purkinje cells inhibit the nucleus cells that are responsible for producing conditioned responses, our simulations suggest a way in which the observed Purkinje cell activity could account for response timing. The conditioned responses permitted by disinhibition of the nucleus cells during the latter parts of the CS would be preceded by periods in which conditioned responses are actively suppressed by increases in simulated Purkinje cell activity. To examine further the role played by increases in Purkinje cell activity during the early parts of the CS, simulations were run under the artificial circumstance in which LTP of gr→Pkj synapses was disengaged during the CS. As shown in Figure 5a, these simulations learned at a comparable rate, but response timing was abnormally broad because the simulated Purkinje cells could only learn to decrease activity during the CS (Fig. 5b,c). Thus, these simulation data suggest the hypothesis that the strong inhibition of nucleus cells that results from learned increases in Purkinje activity suppresses responding early in the CS, thereby sharpening conditioned response timing.
Plasticity at mf→nuc synapses is responsible for the short-latency response observed after lesions of the cerebellar cortex
The hypothesis presented in the last section suggests an explanation for the effects on response retention observed after removal of the cerebellar cortex. Although the original lesion studies focused on lobe HVI of the cerebellar cortex (see Fig. 9) and reported variable effects on conditioned response acquisition and expression (Yeo et al., 1984; Lavond et al., 1987), we have reported previously reliable disruption of conditioned response timing after lesions of the anterior lobe of the cerebellar cortex (see Fig. 9) (Perrett et al., 1993; Garcia and Mauk, 1998). When made in previously trained animals, these lesions produced conditioned responses that were smaller in amplitude, had a fixed, short-latency onset, and could not be modified with further training (see example in Fig.6a). Similarly Figure6b shows that after an extensive lesion (80% of Purkinje cells were removed) of the simulated cerebellar cortex, the appropriate timing of conditioned responses was abolished such that the onset and peak of the response occurred immediately after the presentation of the CS. In agreement with previously reported data, further training after the lesion was incapable of restoring the appropriate timing of conditioned responses (data not shown). Our simulations predict that these seemingly unmodifiable short-latency responses unmasked by lesions of the cerebellar cortex are a result of the LTP that was induced in CS-activated mf→nuc synapses during conditioning. Apparently, these short-latency responses are not observed before the lesion is made because the nucleus cells are being strongly inhibited by the Purkinje cells during the early part of the CS (Fig.4b,c).
Predicted effects of partial lesions of the cerebellar cortex
In contrast to the relatively permanent effects of complete lesions, the simulations make the unusual prediction that partial lesions of the cerebellar cortex should produce a characteristic pattern of postlesion responding that is neither permanent nor a simple dilution of the full effect (Fig. 7). Because the simulations predict that increased Purkinje cell activity suppresses the tendency of the nucleus cell to respond early in the CS (Fig. 4b,c), partial lesions should unmask short-latency responses as well as spare a timed component (Fig. 7a). Moreover, if there is sufficient learning capacity remaining in the spared Purkinje cells, it should be possible with further training to diminish the lesion-induced short-latency component. Indeed Figure7c shows that after a partial lesion that removed 40% of the simulated Purkinje cells, further training produced a large increase in the activity of the remaining Purkinje cells during the early parts of the CS. This increase resulted from additional LTP at gr→Pkj synapses spared by the lesion and active during the early portions of the CS, when climbing fiber activity is being inhibited below its normal level by the short-latency component of the conditioned response. The additional inhibition from the Purkinje cells was eventually able to counteract the excitation provided by the simulated mossy fiber input to the nucleus, thus restoring the prelesion timing of the conditioned response (Fig.7a).
Although the effects shown in Figure 7a are specific to a lesion that removed 40% of the Purkinje cells, we conducted a detailed examination of the effects observed after lesions of different sizes. In general, lesions that removed <20% of the cerebellar cortex had little or no effect on the onset latency of the simulated conditioned response. Lesions that removed 20–40% of the Purkinje cells decreased the onset of the conditioned response, but the short-latency component was smaller than that shown in Figure 7a. As the size of the simulated lesion increased from 40 to 70%, the short-latency component observed after the lesion increased in size, and more postlesion training was required before the timing of the conditioned response was restored. Finally, for lesions that removed >70% of the Purkinje cells, the short-latency component could not be fully extinguished even after prolonged postlesion training. Although it is not possible to infer the precise size of the lesion from our histological analysis, the range of effects displayed by lesions of 20–70% of our simulated Purkinje cells corresponded well with the range of behavioral effects observed after the small electrolytic lesions discussed below.
Testing the predicted partial-lesion effect
We have tested these predictions by making small electrolytic lesions of the anterior lobe of well trained rabbits. We focused on the anterior lobe because, in contrast to the variable effects observed after lesions of lobe HVI (Yeo et al., 1984; Lavond et al., 1987), extensive lesions that included the anterior lobe have resulted previously in reliable disruption of conditioned response timing (Perrett et al., 1993; Perrett and Mauk, 1995). In many of these animals, we observed the predicted pattern of postlesion responding. Figure 7b shows the prelesion and postlesion responding of a representative animal. Before the lesion this animal showed the characteristic adaptively timed conditioned responses. The initial responses after a small electrolytic lesion of the anterior lobe (Fig.7d) show both short-latency and adaptively timed components. Over the course of 6 d of postlesion training, response timing gradually returned to near normal.
We observed this pattern of responding in a total of nine animals with small lesions of the anterior lobe (Figs.8, 9). Although the lesions were not always successful in producing an effect, histological analysis revealed that for most of these failures, the lesion did not affect the anterior lobe of the cerebellar cortex. Thus, in only one case did we observe damage to the anterior lobe that failed to produce a clear disruption in conditioned response timing (Fig. 9,top left NO EFFECT example), and in no case did we observe damage that excluded the anterior lobe but resulted in disrupted timing (Fig. 9). To make quantitative comparisons between the simulated partial-lesion effects and the responding observed in the nine rabbits, we made use of two measures of response timing. The first measure is simply the latency to onset of the conditioned responses. Figure 8a shows the changes in latency to onset for the simulations and the nine rabbits. A one-way repeated measures ANOVA indicated that the lesions produced a significant decrease in latency to onset in the rabbits [F(12,96) = 6.146; p < 0.001]. Subsequent post hoc analysis also showed that the rabbits displayed a significant recovery in latency to onset over the 6 d of postlesion training (Fig. 8a; days indicated by an * are different from baseline, p < 0.05). Separatet test comparisons for each half session reveal only two points where the rabbit data differed significantly from the simulation (Fig. 8a; indicated by ↓, p < 0.05 for at test between a mean and a population). The second measure of response timing used was a ratio of early-response amplitude (initial 200 msec of the CS) and late-response amplitude (during the final 200 msec). Using this measure (Fig. 8b), a score of 1.0 indicates that the short-latency component was as large as the adaptively timed component, whereas a score of 0.0 indicates that there was no short-latency component. Figure 8b shows the similarity in the changes in this early/late ratio for simulated and for rabbit responses. Both groups showed an increase in this ratio after the lesion and then a gradual return to the prelesion value. One-way ANOVA with repeated measures and subsequent post hoc tests revealed (1) a significant increase after the lesions in rabbits [F(12,96) = 6.589;p < 0.001], (2) a significant recovery over the 6 d of training (Fig. 8b; points indicated by an * are different from baseline, p < 0.05), and (3) no significant differences between rabbits and the simulations (p > 0.05 for all time points). Raw traces for two rabbits ordered according to early/late amplitude ratio during different days are shown in Figure 8c, and the sample histology from four animals included in this study are shown in Figure9.
DISCUSSION
We have shown that simulations based on the well characterized synaptic organization of the cerebellum are able to emulate conditioning of properly timed responses. In the simulations, response timing is produced not only by learning to respond at reinforced times but also by learning to suppress responding at incorrect times. This variant of Pavlov's inhibition of delay hypothesis predicts a complex pattern of responding after a partial lesion of the cerebellar cortex. An empirical test of this prediction using small, electrolytic lesions of the cerebellar cortex revealed the pattern of responding predicted by the simulations. Specifically, the initial postlesion responses retained the properly timed component at reinforced times but also showed a new, short-latency component at nonreinforced times. As predicted by the simulations, further training then produced a decrease in the short-latency component, nearly restoring the responses to their original prelesion timing. In contrast to these partial-lesion effects, complete lesions in the simulations reproduced previously published data showing that extensive lesions of the anterior lobe of the cerebellar cortex abolish response timing permanently, even after extensive postlesion training (Perrett and Mauk, 1995).
The simulations suggest that proper response timing is produced by three interacting factors. First, connectivity within the cerebellar cortex produces slight variations in the subsets of granule cells that are active at different times during the CS. This temporal code creates the potential for differential responding at different times during CS. Second, US-activated climbing fiber inputs induce LTD at gr→Pkj synapses active around the time of US presentation. This eventually leads to decreased Purkinje cell activity during the CS, contributing to a conditioned response via disinhibition of the nucleus cells. Alone, this learning would promote relatively poorly timed responses that are inappropriately broad. Thus, of equal importance for adaptive response timing is the induction of LTP at gr→Pkj synapses that are active during the early parts of the CS. In the simulations this extinction-promoting plasticity has two related consequences. It reverses the induction of LTD at gr→Pkj synapses that are equally active at early and late periods of the CS. This helps sharpen the timing of the response by delaying and decreasing the duration of the learned pause in Purkinje cell activity. The induction of LTP at gr→Pkj synapses also allows the cortex to compensate for the learned increase in strength of the mossy fiber synapses, whose activity is quite strong just after CS onset. This latter learning produces increases in Purkinje cell activity that strongly inhibit the nucleus cells early during the CS presentation, also helping to delay the conditioned response until the appropriate time. Thus, the simulations support the idea originally proposed by Pavlov (1927) that adaptive timing could be achieved by learning via extinction not to respond early in the CS as well as by learning via acquisition to respond later in the CS when the US normally occurs.
The potential importance of these two processes is supported by the way in which the behavior of the Purkinje cells in the simulations closely resembles recordings of Purkinje cells during expression of conditioned eyelid responses in ferret (Hesslow and Ivarsson, 1994). The simulated Purkinje cells, like the Purkinje cells recorded by Hesslow and Ivarsson, show an initial increase in activity just after CS onset followed by a decrease that is delayed to produce an adaptively timed response. Although this pattern of activity has not been seen in all studies monitoring Purkinje activity during expression of eyelid responses (Berthier and Moore, 1986; Gould and Steinmetz, 1994), these authors have taken great care to ensure that the recordings were obtained from the region of cerebellar cortex responsible for eyelid movements (Hesslow, 1994).
The simulation results offer potential explanations for the effects produced by both complete and partial lesions limited to the anterior lobe of the cerebellar cortex. Although the effects observed after lesions of lobe HVI of the cerebellar cortex have remained primarily controversial (Yeo et al., 1984), previous studies showed that extensive lesions of the anterior lobe of the cerebellar cortex abolish response timing permanently and yield responses with short latencies (see Fig. 9 for location of these lobes) (Perrett and Mauk, 1995). The present simulation results suggest that the short-latency responses seen after a complete anterior lobe lesion and the short-latency component seen after a partial lesion are caused by the removal of Purkinje cells that have learned, via the induction of LTP, to suppress short-latency responding encouraged by plasticity at the mf→nuc synapses. In the case of a complete lesion, the short-latency responses are relatively unchanged by subsequent training (Perrett and Mauk, 1995). In contrast, we have shown here that with small lesions limited to the anterior lobe of the cerebellar cortex, the short-latency component can be reversed. The simulation results suggest that this compensation is possible as long as there is a sufficient number of Purkinje cells remaining to permit learning of additional suppression of the short-latency component.
The usefulness of a computer simulation is not necessarily related to its ability to produce the proper output, because this can be accomplished in a number of artificial ways. Instead, the goal of the present simulations was to examine how adaptive timing of conditioned responses arises naturally from realistic inputs via interactions between the connectivity of the cerebellar network and bidirectional plasticity at gr→Pkj synapses. This seems in contrast to many previous models that used hypothetical properties specifically designed to accomplish timing. For example, models have used such artificial properties as tapped delay lines (Moore et al., 1989), arrays of CS-activated elements with different time constants (Bullock et al., 1994), or arrays of CS-activated elements that oscillate at different frequencies and phases (Gluck et al., 1990). Instead, our approach has been to build simulations comprised of layers of integrate-and-fire cells whose physiological properties are based on their cerebellar counterparts. These neurons are interconnected according to the known geometry of their projections and their divergence/convergence and numeric ratios. In addition, all simulated cells displayed spontaneous activity consistent with their known physiological firing frequencies, and the activity of cerebellar inputs was based on existing recording data. It is also important that the present results were not peculiar to a particular set of parameters. Although the strength and time course of synaptic connections were carefully matched to electrophysiological data whenever possible, it was still necessary to stipulate values for a few relatively unconstrained parameters. We found that the behavior of the simulations did not change qualitatively across a wide range in these free parameters.
The current simulations represent a significant advance over a less-sophisticated predecessor (Buonomano and Mauk, 1994). The previous version was endowed only with LTD at the gr→Pkj synapses and thus was unable to learn to extinguish or suppress responses during nonreinforced times. For the previous model there were also many more free parameters related to synaptic conductances. To obtain response timing these parameters were set in a way that made granule cell activity vary considerably from one time step to the next. Although this variation allowed responses with appropriately adaptive timing, it also conferred the simulation with a fatal flaw. This tendency to produce extreme variations in granule cell activity also made the network very sensitive to the slightest variation or noise in the CS-activated mossy fiber inputs. Indeed, this network could learn only under the artificial circumstance of perfect trial-to-trial consistency in the activity of the mossy fiber inputs.
In contrast, the present simulation, with its better-constrained synaptic parameters and ability both to learn and to suppress responses, acquires properly timed responses with noisy mossy fiber and climbing fiber inputs based on published recordings (Aitkin and Boyd, 1978; Sears and Steinmetz, 1991). By generating only slight variations in granule activity during the CS, the network is much less affected by noisy inputs. The capacity for learning to suppress responses at nonreinforced times during the CS then sharpens response timing via a process akin to Pavlov's inhibition of delay (Pavlov, 1927). Thus, our approach has been to improve the simulation via the elimination of errors of omission rather than by adding artificial features to make it work—errors of commission. In doing so, the simulation has achieved a level of realism sufficient not only to account for previous findings but also (1) to reproduce naturally the firing characteristics of key cells and (2) to generate complex predictions that are then borne out by experimental test.
APPENDIX: BUILDING A COMPUTER SIMULATION OF THE CEREBELLUM
Network connectivity
The connectivity of the cerebellar cortex was based on the neural network implemented by Buonomano and Mauk (1994). Briefly, there were 104 granule cells, 900 Golgi cells, 600 mossy fiber inputs, 60 basket cells, and 20 Purkinje cells. In scaling the network to computationally feasible dimensions, it was assumed that the convergence/divergence ratios were more important than the cell ratios. Thus, each granule cell received excitatory synaptic inputs from two to six mossy fibers and inhibitory inputs from three Golgi cells. Each Golgi cell received excitatory inputs from 100 granule cells and 20 mossy fibers. Each basket cell received excitatory inputs from 250 granule cells, and each Purkinje cell was contacted by 8000 granule cells and a single climbing fiber. All these connections were randomly chosen from a pool of possible synapses given the known geometry of dendritic spans and maintaining converge/divergence ratios within an order of magnitude. In addition to the cerebellar cortex circuitry described above, this simulation implemented the connectivity of the cerebellar deep nuclei. Thus, the activity of six nucleus cells represented the sole output of the simulation. Each of these output cells received inhibition from 15 Purkinje cells and excitatory inputs from 100 mossy fibers. Finally, the climbing fiber was inhibited by all six nucleus cells. Consistent with anatomical and physiological observations, the circuit was modeled as a closed loop in which the Purkinje cells inhibit nucleus cells that inhibit the climbing fiber providing input to the Purkinje cells (Miall et al., 1998; Voogd and Glickstein, 1998).
Simulation of synaptic potentials
Synaptic current was given by: Equation 1where syn scales synaptic strength,Esyn is the synaptic reversal potential, and gsyn(t) gives the time course of the underlying conductance as expressed by: Equation 2where the summation steps through all presynaptic inputs,Si represents a spike in theith presynaptic input, wi is the synaptic weight of the ith presynaptic input, and τsyn is the decay time constant for the synaptic potential. Thus, synaptic currents were simulated with an instantaneous rise and an exponential decay by summing all inputs of a particular type into a single current that saturates at 1.0 and decays at the rate of τ. The specific values for τ and PSPmax (which gives a more intuitive measure of maximum synaptic strength than does synused in the simulation have been summarized in Table 1. These values were chosen on the basis of the wealth of electrophysiological data that exists for cerebellar synapses (see Table 1, References). In fact there were only two connections (the mossy fiber→Golgi cell and the nucleus→climbing fiber) for which electrophysiological data are currently unavailable. For the results presented here, we used prototype AMPA conductance parameters for the mossy fiber→Golgi cell synapse. However, the conclusions that were drawn from the results did not vary over a wide range of parameters for this connection as long as Golgi cells fired at their known physiological rate of 10–50 Hz. The parameters for the nucleus→climbing fiber synapse could also be varied over a wide range without affecting the results. This is because, as reported previously (Kenyon et al., 1998), this connection is self-regulated via changes in the strength of modifiable synapses to produce the slow background activity of climbing fibers (∼1 Hz).
Simulation of single neurons
All cells were simulated using a standard leaky integrate-and-fire representation based on the idea expressed in Equation 3 that a neuron's membrane potential is determined by summing synaptic and leak conductances: Equation 3where the first term represents the contribution from the leak conductance to the change in membrane potential and the second term summates over all different types of synapses contacting the postsynaptic cell. The only cells that were not represented in this way were the mossy fibers. In the simulation, these input cells do not receive any presynaptic inputs, and thus according to Equation 3 their membrane potential would be stuck at Vrest =El . To simulate the observed range of mossy fiber activity and the inherent variability in firing rate, we depolarized each mossy fiber by an amount taken randomly in each time step from a Gaussian distribution as indicated in Equation 4: Equation 4Each of the 600 mossy fibers could be assigned a preferred rate of activity (range from 10 to 50 Hz) by assigning it a specific distribution mean.
The leaky integrate-and-fire implementation ignores the active membrane properties of neurons, which are dependent on the presence of multiple voltage- and time-dependent ionic currents. Because the detailed voltage-clamp data that would be needed to specify these currents is available only for a few cases, the specification of an appropriate set of parameters would become primarily a matter of conjecture and subject to error. Instead of requiring the specification of all these parameters, the leaky integrate-and-fire neuron fires a spike whenever the membrane potential exceeds a threshold level (θ). The relative refractory period and spike accommodation were simulated by increasing the threshold to a maximum value (θmax) after each spike and allowing it to decay to its initial value with a time constant given by θτ.
Simulation of plasticity
Two sets of connections were modifiable in our simulations. The synapses between granule and Purkinje cells changed according to the following equation:
where δ−gr = −0.00036 and δ+gr = 0.00001 are constants that represent the magnitude of the step decreases and increases in synaptic weight, respectively, GRi is 1 whenever theith granule cell fires and 0 otherwise, and CF(50) is 1 for the 50 msec after a climbing fiber spike and 0 otherwise. Thus, the first term implements the well known rule for LTD by decreasing the weight of the granule synapse by a constant amount when the granule cell is active within 50 msec after a climbing fiber spike. The second term implements LTP at this connection by increasing the weight of the synapse when it is active in the absence of a climbing fiber input in the previous 50 msec (Sakurai, 1987; Hirano, 1990; Salin et al., 1996).
The simulation also implements bidirectional plasticity controlled by Purkinje cell activity at the mossy fiber to cerebellar nucleus synapse (Miles and Lisberger, 1981; Medina and Mauk, 1999). This plasticity rule is given by the following equation:
where δ−mf = −0.000001 and δ+mf = 0.0002 are constants that represent the magnitude of the step decreases and increases in synaptic weight, respectively, and MFi is 1 whenever theith mossy fiber fires and 0 otherwise. ΘLTDPKJ(50) is a binary parameter that equals 1 whenever the average Purkinje cell activity seen in the preceding 50 msec by the postsynaptic nucleus cell increases over a threshold value (∼80 Hz). Similarly, ΘLTPPKJ(50) equals 0 except when average Purkinje cell activity falls below a threshold value (∼40 Hz). The results did not depend on the precise threshold values as long as LTD was induced during higher than normal inhibition of the nucleus by the Purkinje cells and LTP was induced during lower than normal inhibition. Synaptic weights were restricted to the interval [0,1] by preventing further changes in the same direction when the synaptic weights reached 0 or 1.
Simulation of eyelid conditioning
Because CS and US map onto the mossy fiber and climbing fiber pathways, respectively (McCormick et al., 1985; Steinmetz et al., 1985,1988; Mauk et al., 1986; Lewis et al., 1987), these stimuli were simulated by altering the activity of these cells according to empirical data. Mossy fibers are known to be strongly activated either phasically or tonically (in a 3:1 ratio approximately) by tone stimuli (Aitkin and Boyd, 1978) (Fig. 1). Thus, phasic mossy fibers were simulated by randomly choosing 3% of the mossy fibers and increasing the means of their Gaussian distributions (Eq. 4) for the first 20 msec of the CS. In addition, the means for an additional 1% of the mossy fibers (tonic mossy fibers) were increased for the whole duration of the CS. Histograms for these inputs are shown in Figure 1. The activity of the rest of the mossy fibers did not change during the CS. Although the choice of 4% as the number of mossy fibers conveying information about the CS is subject to judgment, we feel that it captures the idea that stimuli will not engage all the mossy fibers to the cerebellum but rather must exist against a background of activity. In general, the more mossy fibers that were engaged by the CS, the quicker learning proceeded. Figure 1 also shows how the US activates the climbing fiber pathway (Sears and Steinmetz, 1991). Thus, to simulate presentation of the US, the climbing fiber was given a step-depolarizing pulse of constant magnitude that lasted 20 msec. This pulse was enough to produce a climbing fiber spike during the initial training trials. However, as training proceeded and climbing fibers began to be inhibited by the increased nucleus activity associated with the conditioned response, the depolarizing pulse was not always successful in producing a climbing fiber spike (Sears and Steinmetz, 1991).
Footnotes
This research was supported by National Institutes of Health Grants MH 57051 and MH 46904.
Correspondence should be addressed to Dr. Michael D. Mauk, Department of Neurobiology and Anatomy, University of Texas Medical School, 6431 Fannin, Houston, TX 77030. E-mail: mmauk{at}nbal9.med.uth.tmc.edu.