Motivated behaviors comprise appetitive actions whose occurrence results partly from an internally driven incentive to act. Such impulsive behavior can also be regulated by external rewarding stimuli that, through learning processes, can lead to accelerated and seemingly automatic, compulsive-like recurrences of the rewarded act. Here, we explored such behavioral plasticity in Aplysia by analyzing how appetitive reward stimulation in a form of operant conditioning can modify a goal-directed component of the animal's food-seeking behavior. In naive animals, protraction/retraction cycles of the tongue-like radula are expressed sporadically with highly variable interbite intervals. In contrast, animals that were previously given a food-reward stimulus in association with each spontaneous radula bite now expressed movement cycles with an elevated frequency and a stereotyped rhythmic organization. This rate increase and regularization, which was retained for several hours after training, depended on both the reward quality and its contingency because accelerated, stereotyped biting was not induced in animals that had previously received a less-palatable food stimulus or had been subjected to nonassociative reward stimulation. Neuronal correlates of these learning-induced changes were also expressed in the radula motor pattern-generating circuitry of isolated buccal ganglia. In such in vitro preparations, moreover, manipulation of the burst frequency of the bilateral motor pattern-initiating B63 interneurons indicated that the regularization of radula motor pattern generation in contingently trained animals occurred separately from an increase in cycle rate, thereby suggesting independent processes of network plasticity. These data therefore suggest that operant conditioning can induce compulsive-like actions in Aplysia feeding behavior and provide a substrate for a cellular analysis of the underlying mechanisms.
- Aplysia feeding
- reward stimulus
- operant conditioning
- buccal ganglia
- radula motor pattern generation
- motor network
Motivated behaviors, such as feeding and sexual activity, constitute an essential part of the motor repertoire of all animals. Such behaviors are often composed of a flexible appetitive phase when animals engage in action patterns that are directed toward obtaining relevant reinforcing stimuli and that are driven unpredictably with no apparent organization in time. This flexibility in the repetitive initiation of appetitive behavior is partly regulated through learning by external sensory stimuli that allow animals to modify their impulse to act according to rewarding contingencies (Cardinal and Everitt, 2004; Kelley, 2004; Balleine, 2005). Among these contingencies, the correlation between the emission of a given motor action (operant) and its outcome (access to reward) plays an essential role in behavioral plasticity. Such associative learning, or operant conditioning, reduces variability in the expression of appetitive behavior not only by favoring appropriate (rewarded) motor action but also by regulating its recurrence in time (Skinner, 1981; Cook and Carew, 1986; Brembs et al., 2002; Everitt and Robbins, 2005). This can therefore lead to an elevated expression of a rewarded action that becomes driven in a stereotyped, rhythmic manner at a relatively high frequency. These acquired changes suggest that contingent stimulation can drastically modify the central neuronal processes that determine the initiation and temporal organization of the underlying motor command. To date, however, little is known about the neural basis of such stereotyped, compulsive-like behavior patterns and how they are conferred by sensory input through operant conditioning.
To address this issue, we developed an operant conditioning paradigm for feeding behavior in Aplysia that is designed to examine how motor learning modifies the animal's impulsion for food seeking and that is amenable to cellular analysis. Previously, it was demonstrated that different aspects of Aplysia feeding can be modified by food intake or buccal sensory nerve stimulation through classical or operant conditioning (Susswein et al., 1986; Lechner et al., 2000a,b; Brembs et al., 2002; Lorenzetti et al., 2006). Insights into the neuronal networks that generate Aplysia feeding, particularly those governing biting movements of the rasp-like radula, have derived from the use of isolated preparations of the buccal ganglia that contain the underlying motor circuitry. Such preparations have already allowed elucidation of how a contingent stimulus, in an analog of operant conditioning, is able to select different buccal motor patterns (Nargeot et al., 1997, 1999a,b; Brembs et al., 2004).
The present study on behaving animals extends these previous investigations by analyzing how natural rewarding stimuli can increase the frequency and regularize the temporal organization of impulsive biting movements of the radula, which in unfed animals consist of sporadically occurring cycles of protraction/retraction. The subsequent use of in vitro buccal ganglia preparations allowed a correlation to be made between these behavioral changes and the acquisition of stereotyped rhythmic bursting in buccal motor pattern-initiating B63 neurons (Hurwitz et al., 1997). Such behavioral plasticity, which persisted for several hours after contingent-reward training, was not simply a consequence of the concomitant increase in rate of B63 bursting and the buccal network patterns that drive radula movements.
Parts of this work have been published previously in abstract form (Petrissans and Nargeot, 2004).
Materials and Methods
Aplysia fasciata were obtained from the Lycée de la Mer (Gujan Mestras, France) and the Laboratoire de Biologie Marine (Arcachon, France) and were maintained at 15°C in filtered artificial sea water (ASW) until use. They were fed daily with fresh seaweed (Ulva lactuca) obtained from the Lycée de la Mer and the Station de Biologie Marine de Roscoff (Roscoff, France). The animals were food deprived for 2 d before experiments to ensure an elevated motivational state for food-seeking behavior during experiments. Although experiments reported here were all conducted during the subjective daytime, it is noteworthy that A. fasciata is a nocturnal species that has been found previously to exhibit significant long-term (>24 h) memory induction only when trained during the subjective night (Lyons et al. 2005). However, consistent with the findings of these authors in diurnal A. californica, the formation of short-term (minutes to several hours) memory by the training protocols used in our study on A. fasciata was presumed to be unaffected by circadian modulation.
In the present study, the beginning of a radula bite cycle was defined as the time at which the protracting radula first becomes visible (Fig. 1A) and was recorded by simultaneous voice and computer keyboard signals using Spike 2 Software (Cambridge Electronic Design, Cambridge, UK). To allow such monitoring, animals were initially transferred into a small (8 dm3) transparent aquarium placed over a mirror and filled with 5 liters of fresh and aerated ASW. The aquarium was placed in a chamber maintained throughout experiments at ∼15°C. Animals were then rested for 30 min before training, which consisted of three successive experimental periods (Fig. 1B): a test period before training, a session in which each animal was subjected to associative or nonassociative food-reward training, and an additional test period after this training. Animals remained in the aquarium throughout the behavioral experiments, and the tank liquid was renewed between each test and training session. Throughout the experimental procedure, the expression of spontaneous food-seeking behavior was optimized by the continuous application (with forceps) of a piece of seaweed (1.5 cm2) to the lips, as close as possible to the jaw (Fig. 1A). Care was taken to avoid this inciting stimulus from being actively brushed against the lips and from being bitten or ingested by the animal. The inciting stimulus, which was renewed before (but not during) each period of the behavioral protocol, was also applied during both test periods. In some experiments (see Fig. 5), the testing procedure was performed by a different experimenter who was unaware of the intervening training protocol to which animal had been subjected. Such blind measurements were found to be statistically indistinguishable from those obtained by a single experimenter who had performed both training and test procedures. This confirmed that subjective influences did not interfere in the collection of data, and therefore measurements obtained by the two methods of observation following a given training procedure were pooled and treated as a single population.
For a given pretraining and posttraining test period, radula movements were measured according to the time taken to generate 100 successive bite cycles or by counting the number of bites occurring over a constant interval of 20 min, whichever came later. Although the occurrence of 100 bite cycles usually took ∼20 min, if necessary the sampling time was extended until a maximum of 40 min had elapsed. On this basis, therefore, statistical analyses were performed on the rate of radula movement cycles (bites per min) counted either in a 20 min interval or during the expression of 100 successive movement cycles in a given test period. Although both analytical procedures gave similar statistical results, all data treatments illustrated in this report derived from measurements made over 100 successive radula bite cycles. Animals (18% of total) that did not express any radula movements during the initial 10 min of the pretraining test period or that expressed <100 bites within the maximum 40 min test interval were excluded from the rest of the experiment. These initial criteria were used to select sufficiently active animals and were not applied during the posttraining test period. Because of this difference, before/after training paired comparisons were not performed in subsequent data analyses.
Training sessions also usually lasted 40 min, and the paradigms differed according to the experimental design and animal group, as indicated in the text and Figures 2, 4, and 5. Basically, in an initial set of experiments, in addition to the presence of the inciting stimulus (see above), each animal was allowed to ingest a “reward” stimulus consisting of either a constant sized (diameter of 5 mm) patch of seaweed (U. lactuca) or a similarly sized piece of cellulose (filter paper hydrated with ASW; Macherey-Nagel, Düren, Germany). Individual patches were presented to the animal with forceps and placed on the radula at its maximal protraction just before the onset of retraction. When taken during a bite, such patches were no longer visible after lip closure or during a subsequent radula movement cycle when protraction again occurred. On this basis, a given patch was considered to be fully ingested within a single radula cycle.
In a second set of experiments, to dissociate contingent from noncontingent influences, a reward stimulus consisting of 20 μl of seaweed juice (obtained by a 1 h maceration of 0.4 g of dried U. lactuca in 10 ml of ASW) was injected directly into the buccal cavity with a calibrated micropipette. The pipette tip was introduced into the buccal cavity just before each injection either at maximal radula protraction when the lips were open (during contingent-reward training) or was pushed gently between the pliant lips if the latter were closed (which was generally the case during noncontingent training when reward stimuli were delivered independently of spontaneous radula movements; see below). This procedure in which a non-alimentary stimulus was repetitively in contact with the lips was never found to elicit aversive behavioral responses such as head withdrawal or rejection. Care was taken to avoid the animal from grasping the pipette tip, and, to prevent any loss of seaweed juice from the buccal cavity, the liquid was ejected only when the lips were closed at the end of radula retraction. Juice delivery was timed either to occurrences of radula bite cycles (in the contingent training procedure) or was independently controlled by a digital metronome (during noncontingent training) so that juice stimuli were delivered at fixed intervals and regardless of the timing of spontaneous bites.
In vitro electrophysiology. After the training period in an additional set of experiments, animals were anesthetized by injection of 50–100 ml of MgCl2 in the hemolymph. The buccal ganglia were isolated and pinned out in a Sylgard-lined Petri dish containing saline with the following composition (in mm): 450 NaCl, 10 KCl, 30 MgCl2(6H2O), 20 MgSO4, 10 CaCl2(2H2O), and 10 HEPES, with the pH adjusted to 7.4 with HCl. The in vitro preparations were maintained at 15°C by means of a Peltier cooling device (Nargeot et al., 1997).
Extracellular recordings and stimulation were made using wire electrodes placed against selected peripheral buccal nerves. Monopolar (recording) and bipolar (stimulation) electrodes were isolated from the bath with Vaseline petroleum jelly. Buccal motor patterns were elicited by monotonic stimulation (0.3 ms pulses of 8.5 V at 2 Hz) of the two bilaterally symmetrical 2,3 nerves (n.2,3) (see Fig. 8A2) (Nargeot et al., 1997). This dual stimulation, which resulted in more robust fictive feeding activity than with unilateral stimulation, was applied via a single photo-isolation unit by a Grass S88 stimulator (Astro-Medical/Grass Instruments, Warwick, RI). In all in vitro experiments, measurements of motor activity did not begin until 10 min after the onset of n.2,3 stimulation.
Intracellular recordings were made from connective tissue-desheathed ganglia with glass microelectrodes filled with 2 m KCH3CO2 (tip resistance of 10–20 MΩ). Signals were amplified by Axoclamp-2B electrometers (Molecular Devices, Palo Alto, CA), visualized on a Tektronix (Marlow, UK) 5113 oscilloscope and digitized by an analog-to-digital converter (CED 1401; Cambridge Electronic Design). The buccal motor pattern-initiating B63 neurons were identified according to their axonal projections in the contralateral cerebrobuccal connective, their mixed chemical and electrical synaptic connections with the contralateral B31/32 neurons, and the phase positions of their activity in the buccal motor pattern (Hurwitz et al., 1997; Susswein et al., 2002).
In the test periods of behavioral experiments, the regularity in recurrence of radula bite cycles was analyzed by event autocorrelation using Spike2 software (Engel et al., 1990). In all cases illustrated here, autocorrelations were made over a fixed number of events (100 bites). The data were then expressed as autocorrelation histograms of 0.5 s bin width over a maximal time range of 30 s.
For in vitro experiments, the same analyses were made using the protraction phase onset in radula motor activity recorded from the I2 nerve. Because the frequency of buccal motor output in isolated preparations decreases rapidly in the several minutes after the onset of n.2,3 stimulation, the analysis of radula motor pattern generation (or fictive biting) began 10 min after the beginning of the inciting stimulation (Nargeot et al., 1997). Preparations (control, 6 of 18; contingent, 4 of 16; noncontingent, 4 of 16) that did not express the 100 cycles required for reliable autocorrelation analysis (see below) during the subsequent 2 h of stimulation were excluded from the analysis. That these excluded preparations were distributed evenly among the different training groups indicated that their low levels of activity was not a specific training-dependent effect (control vs either contingent or noncontingent, p = 0.175; contingent vs noncontingent, p = 1). Moreover, to adjust for any gradual decline in the effectiveness of the n.2,3 stimulation or a general “fatigue” of the in vitro preparation that led to an increase in successive interpattern intervals, this residual trend was estimated by linear regression and eliminated from the dataset by time-series analysis with Statistica software (StatSoft, Tulsa, OK). The rate of fatigue, as provided by the calculated regression slope, was not statistically different among the different animal groups, thereby again excluding the possibility that it was a consequence of the previous behavioral training protocol (control, −0.18 ± 0.6 s per interpattern interval; contingent, −0.78 ± 0.23 s; noncontingent, −0.02 ± 0.17 s; H2 = 4.018). The resulting data were then expressed as autocorrelation histograms of 2.5 s bin width over a range of 150 s. The difference in timescales of the autocorrelation histograms used in the behavioral and cellular analyses reflected the slower occurrences of radula motor patterns in vitro, which were approximately five times less frequent than corresponding biting movements in vivo (Kupfermann, 1974; Nargeot et al., 1997).
Behavioral and buccal ganglion activity were considered as rhythmically recurrent when the corresponding autocorrelation histograms could be fitted (correlation coefficient r ≠ 0 at a significant level of p ≤ 0.05) by a damped sinusoidal Gabor function (Engel et al., 1990; Young et al., 1992; Neuenschwander and Varela, 1993) according to the following: with parameters of amplitude (a), period (b), initial phase lag (c) and time constant (d) ≠ 0 at p ≤ 0.05, an amplitude of at least 10% above offset (a/y0 > 0.1), and a minimum of three peaks >0.8 (Young et al., 1992). Computations of these parameters, fitting data by nonlinear regression, and testing of statistical significance of the parameters and of the correlation coefficient (r) were performed with Sigma Plot software (Systat, Richmond, CA).
Animal group analysis.
Group comparisons were made with nonparametric statistical tests because of the departure from a normal distribution and/or the heterogeneity of variances found within or between groups of data. Comparisons of rates of radula biting movements in vivo (or the underlying motor patterns in vitro) between pairs of dependent data groups were made by the Wilcoxon's signed rank test (W). Similar comparisons between three independent groups of animals (or their corresponding isolated preparations) were made by the Kruskal–Wallis test (H). Post hoc pairwise multiple comparisons were made using the Newman–Keuls multiple range test (q). Additional pairwise comparisons of the proportion of animals (or in vitro preparations) expressing rhythmic radula-related activity (actual movements in vivo, or underlying motor patterns or action potential bursts in B63 neurons in vitro) were made using the Fisher's exact test (P) in unpaired-sample procedures or using the McNemar test (χ2) in paired-sample procedures. All tests were considered significant for a probability level of p ≤ 0.05.
Food intake modifies triggering of radula bite cycles
During food-seeking behavior, Aplysia performs several motor acts in combination, including locomotion, head waving, and cycles of protraction and retraction of the radula (Kupfermann, 1974). In the present study, we focused on the participation of the radula in food seeking because this behavioral component is easily observable and quantifiable (Fig. 1A) (Kupfermann, 1974). In all of our behavioral experiments, food seeking was elicited in the presence of an arousing food stimulus applied in continuous contact to the lips of the animal. This non-ingested inciting stimulus not only enhanced the animal's overall motivational state for food seeking but also helped to orient its head for observation of radula position and movement.
In a first step, to assess the effects of an additional food-reward stimulation (delivered during 40 min training) on subsequent food seeking, we analyzed the rate of radula biting during a ≥20 min test period before training and again 1 h after training in three different groups of animals that differed only in their intervening training protocols (see Materials and Methods) (Fig. 1B). In a “seaweed” group, to ensure an action–stimulus contingency characteristic of operant conditioning during the training period, animals (n = 10) received a patch of seaweed small enough to be bitten and swallowed in a single buccal movement, in association with each spontaneous emission of a radula movement cycle. Because the palatable component of seaweed is mainly attributable to its noncellulose constituents (Carefoot, 1980a,b), in a second “cellulose” group (n = 10 animals), the quality of the ingested stimulus was diminished by using similar sized patches of hydrated cellulose in place of seaweed, which were presented to animals with the same contingent-stimulus paradigm. It is noteworthy that, although devalued, this edible stimulus can be swallowed repetitively without eliciting aversive responses such as a decrease in biting or an induction of rejection behavior (Carefoot, 1980a). Finally, the baseline evolution of food-seeking behavior was assessed in a “control” group of animals (n = 10) that were not presented with any food-reward stimulus during their 40 min training period.
For all three animal groups, the mean rate of biting expressed in the initial test period before training (Fig. 2A) were not significantly different (control, 3.9 ± 0.5 bites/min; seaweed, 4.1 ± 0.6; cellulose, 3.9 ± 0.4; H2 = 0.299), thereby confirming that the different groups were initially in comparable behavioral states. In contrast, 1 h after food-reward training, radula movements were significantly different between the groups (Fig. 2B,C) (H2 = 9.481; p < 0.01), with rate of biting (cycles per minute) in the seaweed-trained animals being substantially increased (Fig. 2B2) compared with either control (Fig. 2B1) or cellulose-trained animals (Fig. 2B3). No significant difference in bite rate was observed between the control and cellulose groups (Fig. 2C). These initial observations therefore indicated that an increase in probability of occurrence of food-seeking radula movements can be induced by natural food stimulation in association with spontaneous bite cycles and, furthermore, that this enduring (≥1 h) behavioral change depended on the quality of the food reward itself. It is also important to note that, during actual radula biting movements, the expression of other behavioral components of food seeking such as locomotion and head waving ceased (Teyke et al., 1990). Thus, although these latter behaviors continued to be elicited after the reward-induced increase in biting frequency, their occurrence was restricted to the now markedly shorter intervals between successive radula movement cycles (see below).
We next analyzed the temporal organization of the more frequent radula biting after appetitive training. It was reported previously that, during active consumption of long strips of seaweed, radula movements are expressed in a rhythmic pattern, i.e., with regular intervals between successive protraction/retraction cycles (Kupfermann, 1974) (but see Lum et al., 2005). During food-seeking behavior that preceded reward stimulation, however, we found that only a small proportion of animals (control, 0 of 10; seaweed, 1 of 10; cellulose, 2 of 10) generated such rhythmic biting (as defined by autocorrelation analysis) and that these proportions were not significantly different among the three animal groups (control vs seaweed, P = 1; seaweed vs cellulose, P = 1; control vs cellulose, P = 0.47). Thus, in the large majority of animals, the intervals between consecutive bite cycles were found to be extremely variable, ranging from a few seconds to several minutes and with no apparent temporal organization (Fig. 2A).
In the control group of animals that had not received any food-reward training, radula bite cycles continued to occur indiscriminately with no evident temporal pattern (Figs. 2B1, 3A1). Such random behavior was also preserved in the group of animals that, during their training, had received a cellulose patch in association with each spontaneous bite cycles (Figs. 2B3, 3A3). In contrast, animals in which natural food stimulation was made contingent on each spontaneous bite now expressed highly structured radula behavior, with a strong tendency for movement cycles to occur in a regularly repeating pattern (Figs. 2B2, 3A2) (group mean cycle period, 6.2 ± 0.8 s). As a consequence, the proportion of animals generating such rhythmic behavior was now significantly higher in the seaweed group compared with either the control (p < 0.02) or the cellulose (p < 0.02) groups (Fig. 3B), both of which remained at levels similar to pretraining control.
Together, these results showed that food-reward stimulation associated with each bite cycle modified the motivational state of Aplysia for food seeking for at least 1 h in at least two ways: first, by increasing the rate of occurrence of this motor act, and second, by reorganizing its expression from apparent randomness to a rhythmically recurring pattern. Moreover, these changes were impaired by devaluing the ingested food stimulus (from seaweed to cellulose), whereas the presence of an inciting stimulus throughout all experiments suggested that the behavioral plasticity derived from the reward stimulation alone.
Contribution of operant-stimulus contingency
We next assessed whether the above modifications in the organization of biting movements were a specific consequence of the contingent association of the rewarding food stimulus with the initiation of each bite cycle (corresponding to operant conditioning) or merely resulted from a nonassociative sensitizing effect of the food. Because the behavioral plasticity induced by seaweed was not similarly produced by cellulose, one might expect that taste rather than the physical presence of the reward stimulus played an essential role in the induction of the motor changes. We therefore tested whether these behavioral modifications could be induced by injecting seaweed juice directly into the buccal cavity (see Materials and Methods) in either association or dissociation with spontaneous bite cycles.
The organization of this experiment, which was similar to that described above (Fig. 1B), involved three new animal groups that were subjected to differing training protocols as schematized in Figure 4. In a “contingent” group of animals (n = 16), to produce a contingent association of the reward stimulus with each bite cycle, a pipette containing seaweed extract was introduced into the buccal mass during radula protraction and 20 μl of juice was ejected when the lips closed and the radula retracted (Fig. 4B). In a second “noncontingent” group, each animal (n = 16) received the juice reward regardless of the timing of spontaneous radula movements, so that no explicit association between stimulus and biting occurred (Fig. 4C). This was achieved by ejecting the extract repetitively at 6 s intervals in correspondence with the mean cycle period of rhythmic biting provided by the Gabor sinusoidal functions that were previously fitted to the data from food-reward trained animals (Fig. 3A2). Moreover, experiments were performed in pairs in which a first animal was submitted to the contingent training protocol then based on this experiment, the same number of reinforcing stimuli was delivered to a second animal in a noncontingent training paradigm. In the latter case, therefore, the length of training sessions was determined by the number of fixed-interval stimuli delivered, which invariably occurred over a shorter period than the equivalent number of spontaneous bite-timed stimuli that had been presented during the 40 min training sessions of contingently rewarded partners. Finally, in a “control” group of animals (n = 16), no rewarding stimulus was provided (Fig. 4A), thereby allowing determination of any effects arising specifically from the inciting stimulus to which all three experimental groups were additionally exposed.
That the initial behavioral states of these three groups of animals were similar was again verified by first comparing both their pretraining biting frequencies and the proportions of animals in which these movements were rhythmically expressed. No significant intergroup differences in either parameter were observed. Mean bite rate per group was as follows: control, 3.5 ± 0.3 cycles/min; contingent, 3.5 ± 0.2; noncontingent, 3.5 ± 0.3; H2 = 0.257. Ratios of animals expressing rhythmicity during 20 min or 100 successive bites were as follows: control, 2 of 16 animals; contingent, 2 of 16; noncontingent, 1 of 16; for all pairwise comparisons, P = 1.
In contrast, 1 h after juice reward training, radula movements differed substantially among these groups (Fig. 5) (H2 = 13.834; p < 0.001), with the mean rate of biting after contingent reinforcement (Fig. 5A2) being significantly higher than in either control (Fig. 5A1) or noncontingent (Fig. 5A3) animals (Fig. 5B). The absence of change in radula behavior in both the noncontingent and control groups, in which the mean bite frequencies during the posttraining test period were not significantly different (Fig. 5B), suggested that the simple repetitive (nonassociative) delivery of seaweed juice was not sufficient to account for the persistent (≥1 h) modification in behavior. Rather, because this behavioral change only occurred in the contingent group indicated that the association of a reward stimulus with each spontaneous bite cycle was the critical parameter underlying the subsequent motor plasticity.
One hour after training, the temporal organization of radula movements was also different between the groups. Whereas bite cycles were still randomly expressed (i.e., with irregular intervals between successive cycles) in control and noncontingent animals (Fig. 6A1,A3), radula movements in the contingently reinforced group again tended to occur in a stereotyped rhythm with a group mean cycle period of 4.8 ± 0.2 s (Fig. 6A2). Thus, a significantly higher proportion of animals that generated rhythmic radula behavior was again found in the contingent group (11 of 16 animals) compared with the control (1 of 16) and noncontingent (1 of 16) groups, which themselves were not significantly different (Fig. 6B). These data therefore further support the conclusion that the acquisition of accelerated rhythmic biting depended specifically on the contingency of the rewarding stimulus with radula protraction during training and did not result from the periodicity of the stimulus delivery itself.
It is important to note that the above behavioral analyses included a subset of trials (n = 7 animals per group) in which the posttraining observations were conducted by a second experimenter who was unaware of the training history of the animal. No significant differences were found between such blind observations and the single experimenter measurements obtained from the remaining animals (n = 9) in each group. Finally, the reward-induced increase in the biting rate occurred at the expense of other motor acts, such as locomotory and head waving movements that were not directly targeted by the reward stimulus. The proportion of the posttraining test periods during which animals diverted their attention from the inciting food stimulus and engaged in these other behaviors was significantly lower in the contingent groups (14.1 ± 5.1%) than in either the control (43.2 ± 10.3%; q3 = 3.777) or noncontingent groups (30.4 ± 3.0%; q2 = 4.156). Again, no significant difference was found between these latter two groups (q2 = 1.446). These findings therefore indicate that the reward-dependent increase in the rate of radula biting did not result from a generalized enhancement of animal motor activity, but rather was specific to the contingently rewarded action.
Retention of the behavioral plasticity
To determine the persistence of the changes in biting behavior induced by contingent reinforcement (Figs. 5B, 6B), an additional series of three groups of animals (n = 5 per group) were treated to the same experimental protocols as in Figure 4, but their radula movements were examined at 4 and 24 h after training. In the intervening rest periods, the animals were not subjected to any explicit food stimulation. The rate of biting in these groups were again statistically indistinguishable before training (bites/min: control, 4.3 ± 0.3; contingent, 4.2 ± 0.2; noncontingent, 5.0 ± 0.4; H2 = 2.480), and, in this experimental population, no animal expressed rhythmic biting in the same pretraining period (control, 0 of 5; contingent, 0 of 5; noncontingent, 0 of 5; for all pairwise comparisons, P = 1).
A similar group analysis of radula movements as that made at 1 h after training (Figs. 5B, 6B) but now, after 4 h, revealed a persistence of the induced behavioral changes. As seen in Figure 7A1, the radula biting rate remained significantly higher in the contingent group than either the control or noncontingent groups (H2 = 10.260; p < 0.01). Moreover, the proportion of animals that expressed biting rhythmicity was also significantly higher among contingently trained animals compared with the other two groups in which no such rhythmicity was observed (Fig. 7A2).
However, an additional analysis of these same animals at 24 h after training no longer revealed any intergroup differences in biting behavior. Radula cycle rates were now statistically indistinguishable among the three groups (H2 = 1.042) (Fig. 7B1), and, similarly, significant differences were no longer found in the proportions of animals that expressed regularized rhythmic biting (Fig. 7B2) (P = 1 for all pairwise comparisons).
These results therefore indicated that the contingent reward-induced changes in radula behavior were retained for at least 4 h under conditions when both the reinforcing and inciting stimuli were withheld but disappeared during the ensuing 20 h. Although establishing the precise time course of this extinction or any circadian influence on long-term memory induction (Lyons et al., 2005) was beyond the scope of the present study, the finding that the behavioral plasticity persisted for several hours offered the possibility of additional analysis at the cellular and network level in vitro.
Neuronal correlates of the behavioral changes
The central neural circuitry that drives radula movements is contained in the bilaterally symmetrical buccal ganglia (Elliott and Susswein, 2002) and is already known to provide a neuronal locus for different forms of operant and classical conditioning of feeding-related behavior both in vivo (Lechner et al., 2000b; Brembs et al., 2002) and in vitro (Nargeot et al., 1999a,b; Mozzachiodi et al., 2003). Therefore, in a first step toward a cellular examination of the behavioral plasticity described in the present study, an additional series of control, contingently and noncontingently trained animals (12 in each case) was used to assess whether the buccal ganglia after isolation in vitro were still able to express essential features of the operant conditioning of radula movements that were previously induced in vivo.
After verifying equivalence in pretraining behavioral status, the animal groups in this new experiment were again subjected to the same training protocols as described in Figure 4 (Fig. 8A1). Immediately after training, their buccal ganglia were isolated (Fig. 8A2) and extracellular recordings were made from peripheral buccal nerves to monitor the protraction (I2 n.), retraction (n.2,1), and closure (R n.) phases of each spontaneous radula motor pattern (Morton and Chiel, 1993; Nargeot et al., 1997). Simultaneous intracellular recordings were also made from the bilaterally symmetrical pair of B63 neurons whose endogenous bursting activity plays a critical role in triggering the buccal output pattern normally responsible for each radula movement cycle in the intact animal (Hurwitz et al., 1997). In these preparations, moreover, monotonic electrical stimulation (2 Hz, 8.5 V) of the bilateral n.2,3 (Fig. 8A2) was used as an in vitro analog of the inciting stimulus applied in vivo (Nargeot et al., 1997) (see Material and Methods). Data collection started 10 min after the onset of the inciting n.2,3 stimulation and continued until 100 consecutive radula motor patterns had occurred over the ensuing 110 min (Fig. 8A1). Thus, each in vitro experiment (from the beginning of dissection) lasted a maximum of ∼4 h, corresponding to the duration of the contingent reinforcement-induced changes in radula behavior observed in vivo (Fig. 7A).
The rate of radula motor pattern generation (or fictive biting) and associated bursts of action potentials in B63 were different among the three groups of isolated preparations, being significantly higher in buccal ganglia originating from the contingent animal group than those taken from either the control or noncontingent groups (Fig. 8B–E) (H2 = 6.260; p < 0.05). Again, no significant difference was found between preparations from the control and noncontingent groups. The numbers of preparations in which radula motor patterns were rhythmically expressed also differed between the groups. Whereas buccal ganglia from control and noncontingent animals produced fictive biting and associated B63 bursts in a random manner with highly variable durations (Fig. 8B,D), the equivalent activity in buccal ganglia from the contingent group was generally expressed in a stereotyped rhythmic pattern (Fig. 8C). This is also evident in the group analysis of Figure 8F, in which a higher proportion of preparations generating such regularized fictive biting and rhythmic B63 bursting was found in the contingent group compared with the control and noncontingent groups, which themselves were not significantly different. These in vitro data therefore indicated that neuronal correlates of the behavioral changes induced by in vivo operant conditioning are indeed expressed both in the underlying buccal network output and at the cellular level by a pair of identified neurons that play a critical role in the initiation of each bite cycle.
The cycle rate increase and regularization involve independent neural processes
It was possible that the operant reward-induced plasticity described above derived from a general increase in excitability of buccal ganglion motor circuitry, which indirectly reduced the variability in interbite intervals by enhancing the frequency of radula motor pattern production. In this way, therefore, the apparent regularization of radula motor output would be merely a secondary consequence of the primary rate increase. Alternatively, the acquisition of rhythmically organized radula behavior could arise from a separate neuronal property that is modified independently of changes in the rate of motor pattern genesis or levels of central network excitability.
To distinguish between these two possibilities, we examined a new series of in vitro preparations from animals belonging to control, contingent and noncontingent groups in which the rate of radula motor pattern generation was experimentally modified by altering the frequency of membrane potential-dependent bursting in the B63 pattern-initiating neurons (Fig. 9). Under continuous inciting stimulation of n.2,3, spontaneous radula output was first recorded in the absence of manipulation of the membrane potential of the B63 neurons, then the rate of buccal network output was either increased or decreased by injecting constant intracellular currents into one of the bilateral B63 cells for periods of 15 min. Such current injection could drastically modify the rate of fictive biting (Fig. 9, middle and top) to the extent that, with sufficient hyperpolarizing current (−5 nA), a complete suppression of motor pattern generation occurred (Fig. 9, bottom) (Hurwitz et al., 1997).
In isolated preparations from control and noncontingently trained animals (n = 7 per group) that continued to generate irregularly timed motor patterns (Fig. 9A,C, middle), depolarizing currents (+2 nA) injected into the B63 cells increased the rate of fictive biting (Fig. 9A,C, top) to a level similar to that observed in preparations from contingently trained animals (Fig. 9B, top) but never induced a regularization of motor pattern production (Fig. 9A–C, compare top panels). Conversely, in preparations (n = 7) from contingently trained animals that already expressed accelerated and regularized fictive biting (Fig. 9B, top), sustained hyperpolarization (−2 nA) of the B63 neurons decreased the frequency of pattern generation to mean levels equivalent to those occurring spontaneously in the other two preparation groups but without altering the regularity of motor pattern genesis (Fig. 9B, middle, compare with A,C, middle). Thus, as seen in the group data analyses of Figure 10, although manipulation of the membrane potential of the B63 motor pattern-initiating neurons induced substantial variability in the rate of fictive biting (Fig. 10A), these changes occurred without significant effect on the preexisting temporal organization of the output patterns themselves (Fig. 10B): fictive biting in buccal ganglia from control and noncontingently trained animals continued to be irregularly expressed, whereas stereotyped rhythmic output persisted in preparations from contingently trained animals. These observations therefore strongly suggested that the regularization of the radula motor command induced by contingent reward training was not simply a secondary consequence of an accelerated rate of motor pattern generation, but rather that the operant learning engaged separate processes of neural plasticity at the network and/or cellular levels.
Operant conditioning and induction of compulsive-like feeding behavior
There is considerable evidence that radula movements, which are organized in an all-or-none action pattern of protraction–retraction mouth closure, constitute a highly variable component of Aplysia feeding behavior (Horn et al., 2004; Jing et al., 2004; Jing and Weiss, 2005; Lum et al., 2005; Zhurov et al., 2005; Ye et al., 2006). We now show that the temporal structure of radula bite cycles during food seeking is also a flexible constituent of this behavior. In the absence of palatable-food ingestion, animals expressed infrequent bite cycles separated by highly variable intervals at which time other actions occurred, including head turning and locomotion that were previously found to participate in the animal's appetitive behavior (Kupfermann, 1974; Teyke et al., 1990; Kupfermann et al., 1991). After a period of contingent food-reward training, however, the variability in occurrence of radula movements was drastically reduced, leading to prolonged (for several hours) bouts of intense and seemingly automatic rhythmic biting. In contrast, the expression of accompanying head waving and locomotor activity decreased. This concomitant decline in occurrence of these other appetitive acts therefore indicated that the increased rate and regularization of biting was an explicit behavioral plasticity rather than a consequence of a generalized increase in the motivational state of the animal.
Several arguments suggest that the plasticity in radula behavior depended primarily on action–reward contingency required in operant conditioning. First, the changes specific to the radula motor command did not merely reflect behavioral responses analogous to a motor routine or habit (Serruya and Eilam, 1996; Balleine, 2005). Unlike the “purposelessness” of a habitual process, the induction of rhythmic radula biting was closely dependent on the nature of food stimulation because the motor plasticity did not occur under delivery of cellulose instead of seaweed, and it was extinguished under continuous inciting stimulation after the reward stimulus was withheld. Moreover, because the behavioral changes did not develop in control animals that were subjected to inciting stimulation in the absence of repetitive food reward indicated that the motor plasticity resulted neither from a global sensitizing effect of the inciting stimulation nor from a simple strengthening in the relationship between the inciting stimulus and the elicited radula action in a manner equivalent to a habitual learning process (Everitt and Robbins, 2005; Faure et al., 2005).
Rather, a strict contingence of the appetitive reward with motor action appeared to be necessary and sufficient for the induction of the behavioral plasticity, because the changes in radula operation did not develop in a noncontingent paradigm of intrabuccal food stimulation, i.e., when repetitive delivery of the reinforcing seaweed extract was uncorrelated with spontaneous bite cycles. Thus, stereotyped rhythmic biting did not simply arise from a nonassociative effect (such as sensitization) of the rewarding stimulus or from classical conditioning, an associative form of learning that could develop through an inciting and rewarding stimulus contingency (Wyvell and Berridge, 2000, 2001).
The posttraining rhythmic radula movements expressed by Aplysia were therefore “purposeful” and directed specifically toward obtaining a food reward, even when this appetitive stimulus was no longer available. Such perseverance of an accelerated and stereotyped yet ultimately unsuccessful (i.e., unrewarded) behavior that also dominates other motor actions shares essential features with the elevated and automatically recurring motor acts in the compulsive behavior patterns of other animals (Pitman, 1989; Serruya and Eilam, 1996; Eilam et al., 2006). The compulsion to act has been described in a variety of nonpathological goal-directed behaviors, including feeding and sexual behavior and often results from persistent changes in a specific component of a subject's behavior after an associative reward stimulus has been withdrawn (Wyvell and Berridge, 2000, 2001; Joel and Avisar, 2001; Belozertseva et al., 2004; Joel et al., 2005).
Learning paradigms that use contingent reinforcement in the form of operant conditioning have been found to modify a variety of invertebrate behaviors, including foraging and communication in insects (Hammer and Menzel, 1995), and exploratory (Fitzgerald et al., 1997) and respiratory (Spencer et al., 1999) behaviors in mollusks. Moreover, operant (aversive) learning has been recently described in the Aplysia gill- and siphon-withdrawal reflex, the most extensively studied cellular model of associative and nonassociative learning (Hawkins et al., 2006). In freely feeding Aplysia, the explicit association of a food stimulus (Susswein et al., 1986) or corresponding sensory stimulation (Brembs et al., 2002) with the emission of spontaneous radula movements has been reported previously to enhance the overall frequency of subsequent cycles. In addition to confirming these observations, our results further show that such contingent reward reinforcement in operant conditioning can confer a long-lasting temporal reorganization of an otherwise randomly occurring behavior into producing stereotyped and compulsive-like repetitive actions. The present results therefore constitute the first description of such compulsive-like behavior in Aplysia induced by operant learning and, importantly, offer a simple system in which the neuronal plasticity that underlies the induction and maintenance of rhythmically recurring action in a motivated behavior can be determined.
Cellular loci for operant learning
Numerous key elements of the buccal networks that orchestrate Aplysia radula movements have now been identified, and their synaptic connectivity and bioelectrical properties have been described. These feeding networks have also been found to be the target sites for other types of functional plasticity induced by associative learning both in vivo and via equivalent paradigms of sensory nerve stimulation in vitro (Nargeot et al., 1999a,b; Lechner et al., 2000b; Mozzachiodi et al., 2003; Lorenzetti et al., 2006) and therefore offer the most likely cellular loci for the behavioral plasticity described in the present study.
Two changes in feeding network activity of isolated buccal ganglia were closely correlated to the behavioral changes induced by previous operant conditioning in vivo: (1) an increase in the mean frequency of spontaneously emitted radula motor patterns, and (2) a change in their organization in time. Furthermore, equivalent modifications were found in the burst generating properties of two feeding network elements, the previously identified pair of B63 interneurons. The functional relevance of B63 in the present context is important, because this interneuron is an endogenously bursting cell whose activity precedes each cycle of buccal circuit output and is essential for instigating the motor program responsible for each bite cycle (Hurwitz et al., 1997). The B63 neuron is therefore a plausible site for the operant learning described here and suggests that such functional plasticity not only involves network components (such as neuron B51) responsible for selecting between different buccal motor programs (Nargeot et al., 1999a,b; Brembs et al., 2002) but may extend to other circuit elements (like B63) whose task is to provide the command that actually triggers these programs. This in turn adds a new dimension to the mechanistic basis of operant conditioning in Aplysia feeding by suggesting that the neural plasticity underlying modifications in the initiation (e.g., B63) and specification (e.g., B51) of radula motor action are encoded at different cellular loci.
The ability to modify the rate of motor pattern genesis by current injection into B63 without altering the temporal organization of these patterns suggests that the acquisition of stereotyped rhythmicity did not simply result from a generalized change in excitability of the motor pattern-generating circuit but, rather, indicates that the selective modification of neuronal bursting properties was involved. Indeed, it is well known that individual neurons can express either irregular or regular forms of burst firing as a result of specific sets of voltage-dependent conductances and intracellular signal cascades that can be activated and modified by neuromodulatory transmitters (Canavier et al., 1994; Turrigiano et al., 1994; Lechner et al., 1996; Falcke et al., 2000). Although in the context of the present study the pattern-initiating B63 neurons themselves are obvious candidates for such modulatory control, the possible contribution of modifications to the burst-generating properties of other buccal pattern-generating neurons to the learning-induced changes in network activity cannot be excluded.
Alternatively (or conjointly), the behavioral plasticity may derive from an underlying network mechanism involving changes in the synaptic interactions between specific subsets of buccal central pattern-generating circuitry that include the B63 neurons. Here again, there is compelling evidence from other neural systems indicating that synaptic connectivity within neuronal assemblages, particularly inhibitory synapses and electrical coupling, provide flexible substrates for switching neural networks between random and regular rhythmic modes of bursting activity (Lewis and Rinzel, 2000; Szucs et al., 2000; Urbain et al., 2002; Staras et al., 2003). Clearly, additional analysis of the membrane properties of the B63 neurons and their synaptic relationships with other members of the radula pattern-generating circuit is now required to determine the possible contributions of such mechanisms to operant learning in Aplysia feeding behavior.
This work was supported by the “Conseil Régional d'Aquitaine/Fonds Européens de Développement Régional.” We thank Dr. M. Le Bon-Jego for testing the animals using a blind procedure and Drs. D. Combes, L. Juvin, D. Le Ray, and D. Morin for helpful discussions.
- Correspondence should be addressed to Dr. Romuald Nargeot, Laboratoire Mouvement Adaptation Cognition, Centre National de la Recherche Scientifique, Unité Mixte de Recherche 5227, Université Bordeaux 2, Zone Nord, Bâtiment 2A, 146 rue Léo Saignat, 33076 Bordeaux, France.