Abstract
Associative learning in goal-directed behaviors, in contrast to reflexive behaviors, can alter processes of decision-making in the selection of appropriate action and its initiation, thereby enabling animals, including humans, to gain a predictive understanding of their external environment. In the mollusc Aplysia, recent studies on appetitive operant conditioning in which the animal learns about the positive consequences of its behavior have provided insights into this form of associative learning which, although ubiquitous, remains mechanistically poorly understood. The findings support increasing evidence that central circuit- and cell-wide sites other than chemical synaptic connections, including electrical coupling and membrane conductances controlling intrinsic neuronal excitability and underlying voltage-dependent plateauing or oscillatory mechanisms, may serve as the neural substrates for behavioral plasticity resulting from operant conditioning. Aplysia therefore continues to provide a model system for understanding learning and memory formation that enables establishing the neurobiological links between behavioral, network, and cellular levels of analysis.
Similar content being viewed by others
References
Byrne JH, Baxter DA, Buonomano DV, Cleary LJ, Eskin A, Goldsmith JR, McClendon E, Nazif FA, Noel F, Scholz KP (1991) Neural and molecular bases of nonassociative and associative learning in Aplysia. Ann N Y Acad Sci 627:124–149
Glanzman DL (2006) The cellular mechanisms of learning in Aplysia: of blind men and elephants. Biol Bull 210:271–279
Hawkins RD, Kandel ER, Bailey CH (2006) Molecular mechanisms of memory storage in Aplysia. Biol Bull 210:174–191
Reynolds GS (1975) A primer of operant conditioning, 2nd edn. Scott Foresman, Glenview
Thorndike EL (1911) Animal intelligence: experimental studies. Macmillan, New York
Skinner BF (1938) The behavior of organisms: an experimental analysis. Appleton-Century-Crofts, New York
Cook DG, Carew TJ (1986) Operant conditioning of head-waving in Aplysia. Proc Natl Acad Sci USA 83:1120–1124
Susswein AJ, Schwarz M, Feldman E (1986) Learned changes of feeding behavior in Aplysia in response to edible and inedible foods. J Neurosci 6:1513–1527
Skinner BF (1981) Selection by consequences. Science 213:501–504
Brembs B (2003) Operant conditioning in invertebrates. Curr Opin Neurobiol 13:710–717
Everitt BJ, Robbins TW (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 8:1481–1489
Pitman RK (1989) Animal models of compulsive behavior. Biol Psychiatry 26:189–198
Cardinal RN, Everitt BJ (2004) Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Curr Opin Neurobiol 14:156–162
Kelley AE (2004) Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev 27:765–776
Balleine BW (2005) Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav 86:717–730
Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ (2009) Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res 199:89–102
Balleine BW, Delgado MR, Hikosaka O (2007) The role of the dorsal striatum in reward and decision-making. J Neurosci 27:8161–8165
Hoyle G (1980) Learning, using natural reinforcement, in insect preparations that permit cellular neuronal analysis. J Neurobiol 11:323–354
Wustmann G, Rein K, Wolf R, Heisenberg M (1996) A new paradigm for operant conditioning of Drosophila melanogaster. J Comp Physiol A 179:429–436
Spencer GE, Syed NI, Lukowiak K (1999) Neural changes after operant conditioning of the aerial respiratory behavior in Lymnaea stagnalis. J Neurosci 19:1836–1843
Gardner D (1977) Interconnections of identified multiaction interneurons in buccal ganglia of Aplysia. J Neurophysiol 40:349–361
Susswein AJ, Byrne JH (1988) Identification and characterization of neurons initiating patterned neural activity in the buccal ganglia of Aplysia. J Neurosci 8:2049–2061
Plummer MR, Kirk MD (1990) Premotor neurons B51 and B52 in the buccal ganglia of Aplysia californica: synaptic connections, effect on ongoing motor rhythms, and peptide modulation. J Neurophysiol 63:539–558
Hurwitz I, Susswein AJ (1996) B64, a newly identified central pattern generator element producing a phase switch from protraction to retraction in buccal motor programs of Aplysia californica. J Neurophysiol 75:1327–1344
Hurwitz I, Kupfermann I, Susswein AJ (1997) Different roles of neurons B63 and B34 that are active during the protraction phase of the buccal motor programs in Aplysia californica. J Neurophysiol 78:1305–1319
Kabotyanski EA, Baxter DA, Byrne JH (1998) Identification and characterization of catecholaminergic neuron B65, which initiates and modifies patterned activity in the buccal ganglia of Aplysia. J Neurophysiol 79:605–621
Jing J, Weiss KR (2002) Interneuronal basis of the generation of related but distinct motor programs in Aplysia: implications for current neuronal models of vertebrate interlimb coordination. J Neurosci 22:6228–6238
Jing J, Cropper EC, Hurwitz I, Weiss KR (2004) The construction of movement with behavior-specific and behavior-independent modules. J Neurosci 24:6315–6325
Elliott CJH, Susswein AJ (2002) Comparative neuroethology of feeding control in molluscs. J Exp Biol 205:877–896
Kupfermann I (1974) Feeding behavior in Aplysia: a simple system for the study of motivation. Behav Biol 10:1–26
Nargeot R, Petrissans C, Simmers J (2007) Behavioral and in vitro correlates of compulsive-like food-seeking induced by operant conditioning in Aplysia. J Neurosci 27:8059–8070
Botzer D, Markovich S, Susswein AJ (1998) Multiple memory processes following training that a food in inedible in Aplysia. Learn Mem 5:204–219
Alberini CM, Ghirardi M, Metz R, Kandel ER (1994) C/EBP is an immediate-early gene required for the consolidation of long-term facilitation in Aplysia. Cell 76:1099–1114
Taubenfeld SM, Milekic MH, Monti B, Alberini CM (2001) The consolidation of new but not reactivated memory requires hippocampal C/EBPβ. Nat Neurosci 4:813–818
Levitan D, Lyons LC, Perelman A, Green CL, Motro B, Eskin A, Susswein AJ (2008) Training with inedible food in Aplysia causes expression of C/EBP in the buccal but not cerebral ganglion. Learn Mem 15:412–416
Brembs B, Lorenzetti FD, Reyes FD, Baxter DA, Byrne JH (2002) Operant reward learning in Aplysia: neuronal correlates and mechanisms. Science 296:1706–1709
Lechner HA, Baxter DA, Byrne JH (2000) Classical conditioning of feeding in Aplysia: I. Behavioral Analysis. J Neurosci 20:3369–3376
Nargeot R, Baxter DA, Byrne JH (1997) Contingent-dependent enhancement of rhythmic motor patterns: an in vitro analog of operant conditioning. J Neurosci 17:8093–8105
Nargeot R, Baxter DA, Byrne JH (1999) In vitro analog of operant conditioning in Aplysia. I. Contingent reinforcement modifies the functional dynamics of an identified neuron. J Neurosci 19:2247–2260
Nargeot R, Baxter DA, Byrne JH (1999) In vitro analogue of operant conditioning in Aplysia. II. Modifications of the functional dynamics of an identified neuron contribute to motor pattern selection. J Neurosci 19:2261–2272
Brembs B, Baxter DA, Byrne JH (2004) Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation. Learn Mem 11:412–420
Lechner HA, Baxter DA, Byrne JH (2000) Classical conditioning of feeding in Aplysia: II. Neurophysiological correlates. J Neurosci 20:3377–3386
Lorenzetti FD, Mozzachiodi R, Baxter DA, Byrne JH (2006) Classical and operant conditioning differentially modify the intrinsic properties of an identified neuron. Nat Neurosci 9:17–19
Lorenzetti FD, Baxter DA, Byrne JH (2008) Molecular mechanisms underlying a cellular analog of operant reward learning. Neuron 59:815–828
Mozzachiodi R, Lorenzetti FD, Baxter DA, Byrne JH (2008) Changes in neuronal excitability serve as a mechanism of long-term memory for operant conditioning. Nat Neurosci 11:1146–1148
Morton DW, Chiel HJ (1993) In vivo buccal nerve activity that distinguishes ingestion from rejection can be used to predict behavioral transitions in Aplysia. J Comp Physiol A 172:17–32
Briggman KL, Abarbanel HDI, Kristan WB (2005) Optical imaging of neuronal populations during decision-making. Science 307:896–901
Kristan WB (2008) Neuronal decision-making circuits. Curr Biol 18:R928–R932
Schall JD (2005) Decision making. Curr Biol 15:R9–R11
Nargeot R, Baxter DA, Patterson GW, Byrne JH (1999) Dopaminergic synapses mediate neuronal changes in an analogue of operant conditioning. J Neurophysiol 81:1983–1987
Mozzachiodi R, Byrne JH (2010) More than synaptic plasticity: role of nonsynaptic plasticity in learning and memory. Trends Neurosci 33:17–26
Schwarz M, Susswein AJ (1986) Identification of the neural pathway for reinforcement of feeding when Aplysia learn that food is inedible. J Neurosci 6:1528–1536
Horn CC, Zhurov Y, Orekhova IV, Proekt A, Kupfermann I, Weiss KR, Brezina V (2004) Cycle-to-cycle variability of neuromuscular activity in Aplysia feeding behavior. J Neurophysiol 92:157–180
Reyes FD, Mozzachiodi R, Baxter DA, Byrne JH (2005) Reinforcement in an in vitro analog of appetitive classical conditioning of feeding behavior in Aplysia: blockade by a dopamine antagonist. Learn Mem 12:216–220
Nargeot R, Cornet C, Simmers J (2008) Dopaminergic modulation of oscillatory membrane properties of buccal motor-pattern initiating neurons and their electrical coupling in Aplysia. FENS Abstr 4:021.13
Balleine BW, Liljeholm M, Ostlund SB (2009) The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res 199:43–52
Kemenes G, Staras K, Benjamin PR (2001) Multiple types of control by identified interneurons in a sensory-activated rhythmic motor pattern. J Neurosci 21:2903–2911
Nargeot R, Le Bon-Jego M, Simmers J (2009) Cellular and network mechanisms of operant learning–induced compulsive behaviour in Aplysia. Curr Biol 19:975–984
Chay TR (1996) Electrical bursting and luminal calcium oscillation in excitable cell models. Biol Cybern 75:419–431
Selverston AI, Rabinovich MI, Abarbanel HDI, Elson R, Szücs A, Pinto RD, Huerta R, Varona P (2000) Reliable circuits from irregular neurons: a dynamical approach to understanding central pattern generators. J Physiol (Paris) 94:357–374
Varona P, Torres JJ, Huerta R, Abarbanel HDI, Rabinovich MI (2001) Regularization mechanisms of spiking-bursting neurons. Neural Netw 14:865–875
Falcke M, Huerta R, Rabinovich MI, Abarbanel HDI, Elson RC, Selverston AI (2000) Modeling observed chaotic oscillations in bursting neurons: the role of calcium dynamics and IP3. Biol Cybern 82:517–527
Lewis TJ, Rinzel J (2000) Self-organized synchronous oscillations in a network of excitable cells coupled by gap junctions. Network 11:299–320
Placantonakis DG, Bukovsky AA, Aicher SA, Kiem H-P, Welsh JP (2006) Continuous electrical oscillations emerge from a coupled network: a study of the inferior olive using lentiviral knockdown of connexin36. J Neurosci 26:5008–5016
Lasater EM (1987) Retinal horizontal cell gap junctional conductance is modulated by dopamine through a cyclic AMP-dependent protein kinase. Proc Natl Acad Sci USA 84:7319–7323
Yang XD, Korn H, Faber DS (1990) Long-term potentiation of electrotonic coupling at mixed synapses. Nature 348:542–545
Johnson BR, Peck JH, Harris-Warrick RM (1993) Amine modulation of electrical coupling in the pyloric network of the lobster stomatogastric ganglion. J Comp Physiol A 172:715–732
Kokarovtseva L, Jaciw-Zurakiwsky T, Mendizabal Arbocco R, Frantseva MV, Perez Velazquez JL (2009) Excitability and gap junction-mediated mechanisms in nucleus accumbens regulate self-stimulation reward in rats. Neuroscience 159:1257–1263
Schultz W (1997) Dopamine neurons and their role in reward mechanisms. Curr Opin Neurobiol 7:191–197
Schultz W (2010) Dopamine signals for reward value and risk: basic and recent data. Behav Brain Funct 23:6–24
Wise RA, Rompre PP (1989) Brain dopamine and reward. Annu Rev Psychol 40:191–225
Wise RA (2004) Dopamine and food reward: back to the elements. Am J Physiol Regul Integr Comp Physiol 286:R13
Di Chiara G, Bassareo V (2007) Reward system and addiction: what dopamine does and doesn’t do. Curr Opin Pharmacol 7:69–76
Dalley JW, Everitt BJ (2009) Dopamine receptors in the learning, memory and drug reward circuitry. Semin Cell Dev Biol 20:403–410
Fulton S (2010) Appetite and reward. Front Neuroendocrinol 31:85–103
Olds J, Milner P (1954) Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47:419–427
Le Moal M, Simon H (1991) Mesocorticolimbic dopaminergic network: Functional and regulatory roles. Physiol Rev 71:155–234
Ettenberger A (1989) Dopamine, neuroleptics and reinforced behavior. Neurosci Biobehav Rev 13:105–111
Smith-Roe SL, Kelley AE (2000) Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. J Neurosci 20:7737–7742
Liu Z, Richmond BJ, Murray EA, Saunders RC, Steenrod S, Stubblefield BK, Montague DM, Ginns EI (2004) DNA targeting of rhinal cortex D2 receptor protein reversibly blocks learning of cues that predict reward. Proc Natl Acad Sci USA 101:12336–12341
Fadok JP, Dickerson TMK, Palmiter RD (2009) Dopamine is necessary for cue-dependent fear conditioning. J Neurosci 29:11089–11097
Martínez-Rubio C, Serrano GE, Miller MW (2009) Localization of biogenic amines in the foregut of Aplysia californica: catecholaminergic and serotonergic innervation. J Comp Neurol 514:329–342
Ascher P (1972) Inhibitory and excitatory effects of dopamine on Aplysia neurons. J Physiol (London) 225:173–209
Swann JW, Sinback CN, Carpenter DO (1978) Evidence for identified dopamine motor neurons to the gill of Aplysia. Neurosci Lett 10:275–280
Wright NJ, Walker RJ (1984) The possible site of action of 5-hydroxytryptamine, 6-hydroxytryptamine, tryptamine and dopamine on identified neurons in the central nervous system of the snail, Helix aspersa. Comp Biochem Physiol C 78:217–225
Teyke T, Rosen SC, Weiss KR, Kupfermann I (1993) Dopaminergic neuron B20 generates rhythmic neuronal activity in the feeding motor circuitry of Aplysia. Brain Res 630:26–237
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Nargeot, R., Simmers, J. Neural mechanisms of operant conditioning and learning-induced behavioral plasticity in Aplysia . Cell. Mol. Life Sci. 68, 803–816 (2011). https://doi.org/10.1007/s00018-010-0570-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00018-010-0570-9