Social modulation of learned behavior by dopamine in the basal ganglia: Insights from songbirds

https://doi.org/10.1016/j.jphysparis.2012.09.002Get rights and content

Abstract

Dysfunction of the dopaminergic system leads to motor, cognitive, and motivational symptoms in brain disorders such as Parkinson’s disease. The basal ganglia (BG) are involved in sensorimotor learning and receive a strong dopaminergic signal, shown to play an important role in social interactions. The function of the dopaminergic input to the BG in the integration of social cues during sensorimotor learning remains however largely unexplored. Songbirds use learned vocalizations to communicate during courtship and aggressive behaviors. Like language learning in humans, song learning strongly depends on social interactions. In songbirds, a specialized BG–thalamo-cortical loop devoted to song is particularly tractable for elucidating the signals carried by dopamine in the BG, and the function of dopamine signaling in mediating social cues during skill learning and execution. Here, I review experimental findings uncovering the physiological effects and function of the dopaminergic signal in the songbird BG, in light of our knowledge of the BG–dopamine interactions in mammals. Interestingly, the compact nature of the striato-pallidal circuits in birds led to new insight on the physiological effects of the dopaminergic input on the BG network as a whole. In singing birds, D1-like receptor agonist and antagonist can modulate the spectral variability of syllables bi-directionally, suggesting that social context-dependent changes in spectral variability are triggered by dopaminergic input through D1-like receptors. As variability is crucial for exploration during motor learning, but must be reduced after learning to optimize performance, I propose that, the dopaminergic input to the BG could be responsible for the social-dependent regulation of the exploration/exploitation balance in birdsong, and possibly in learned skills in other vertebrates.

Highlights

► Songbirds are a good model to study the function of dopamine in the basal ganglia. ► Dopamine modulates information transmission through the song-related basal ganglia. ► Dopaminergic input to the basal ganglia modulates song with social context. ► The tonic dopaminergic input may regulate the exploration–exploitation balance.

Introduction

The neuromodulator dopamine (DA) is delivered to a variety of forebrain structures by neurons located in the substantia nigra par compacta (SNc) and ventral tegmental area (VTA) (Smith and Kieval, 2000). Dysfunction of the dopaminergic system in Parkinson’s disease or drug addiction leads to motor, cognitive, and motivational symptoms (Barbeau, 1967, Hoehn and Yahr, 1967, Wise, 1987, Sulzer, 2011). Although it has long been identified with motor function, DA is therefore also essential for cognitive and motivational processes (Salamone, 1992, Wise, 2004). In particular, the involvement of DA in motivation and reward learning has been emphasized in the last decades.

Behaviors resulting in the obtaining rewards, such as sexual behavior, social behavior, feeding, and foraging, are likely to be repeated, reflecting a high degree of motivation of the subject. Such rewards are advantageous or necessary for the survival of individuals, and reward learning is central to the implementation of the interactions of individuals between themselves (social interactions) and with their environment. Because rewarding events, such as food consumption, copulation, and drug uptake, are associated with strong DA release (Schultz et al., 1993), DA neurons are thought to code for the rewarding value of events. In the framework of reinforcement learning theory (Sutton and Barto, 1990), DA delivery in the striatum reflects reward prediction errors and provides a reinforcement signal guiding motor learning toward optimal behaviors, eventually leading to maximal reward uptakes (Montague et al., 1996, Phillips et al., 2003, Wise, 2004). It important to note, however, that DA is neither necessary nor sufficient to mediate the hedonic impact of reward (Berridge, 2007), and that other brain regions may encode reward and its prediction (Gonon, 2009), questioning the simplistic view of DA as a reward signal.

The extrasynaptic distribution of DA receptors indicates that DA neurons convey information by volume transmission (one-to-many) in the extracellular space rather than by one-to-one synaptic transmission (Zoli et al., 1999, Gonon et al., 2000). In the view of DA as a reward signal, fast phasic changes in extracellular DA concentration activity on a subsecond to second timescale encode the probability of reward availability, its uncertainty, and more generally the prediction error (Schultz et al., 1997, Fiorillo et al., 2003). But importantly, DA neurons can operate in distinct temporal modes (Hauber, 2010). On one hand, DA neurons display phasic firing in bursts of action potentials in relation to behaviorally relevant events. Such phasic bursts are mediated by glutamatergic inputs originating in particular in the ventral tegmentum, gated by inhibitory afferents from the ventral pallidum (VP) (Grace et al., 2007). Burst firing triggers high amplitude, transient DA release. On the other hand, DA neurons display slow and irregular spontaneous activity (2–10 Hz). Although it is driven by an intrinsic pacemaker, spontaneous activity is regulated by afferent input, in particular the inhibitory connections from the VP (Grace and Bunney, 1984). The modulation of this tonic firing of DA neurons underlies slow changes in the concentration of tonic extracellular DA on much slower time scales (seconds to minutes) (Floresco et al., 2003). Spillover from synapses following phasic DA release also contributes to modulate extrasynaptic DA concentrations and is tightly regulated by DA transporters (Cragg and Rice, 2004).

These temporally distinct modes of DA signaling seem to be related to different kinds of information and may serve dissociable behavioral functions. Slower changes in tonic DA levels in the basal ganglia (BG) and cortex likely serve functions other than signaling reinforcement in motor learning (Salamone et al., 2005, Costa, 2007). Interestingly, reward processing and social interaction processes share common neural substrates (Caldú and Dreher, 2007). One might consider the possibility that phasic DA release in the striatum signals reward prediction error, and may provide a reinforcement signal to the BG (Schultz et al., 1993, Phillips et al., 2003), while tonic dopaminergic signaling would be involved in the representation of motivational state related to social cues and contexts (Wang et al., 1999, Anstrom et al., 2009, Aragona and Wang, 2009). Another possibility is for tonic DA signals to be involved in the regulation of inter-individual differences in exploration and exploitation behaviors (Frank et al., 2009). The function of tonic DA levels in the BG, outside of specific pathophysiology, remains largely unexplored.

Songbirds use learned vocalizations to communicate during courtship or aggressive behaviors. These vocalizations, called song, represent a complex sensorimotor task and require fast coordination of the numerous laryngeal and respiratory muscles. Moreover, song learning in young birds relies on a variety of complex social cues which may serve to open some attentional or arousal gate, which then permits sensory learning (Doupe and Kuhl, 1999). For example, songbirds have been shown to learn alien songs from live tutors when they would reject the same songs presented by tape playback (Petrinovich and Baptista, 1987). In adult birds, song performance is also modulated by social context, and male birds sing a more stereotyped song when singing to a female than when singing alone (Sossinka and Böhner, 1980, Kao et al., 2005). As DA has been involved in the control of both complex motor (Wickens et al., 2003, Joshua et al., 2009) and social behaviors (Young and Wang, 2004, Skuse and Gallagher, 2009), songbirds offer a unique opportunity as a model system to study the role of DA signaling in the social modulation of a learned sensorimotor task.

Over the past 15 years, advances in anatomical, physiological and histochemical characterization of avian BG neurons and circuitry have revealed evolutionary homologies to mammalian BG networks (Fig. 1; Bottjer and Johnson, 1997, Reiner et al., 1998, Luo et al., 2001, Reiner et al., 2004, Doupe et al., 2005). The song-related BG nucleus Area X contains neuron types homologue to both the striatal and pallidal components of the mammalian BG circuit (Farries and Perkel, 2002, Goldberg and Fee, 2010, Goldberg et al., 2010). It receives dense dopaminergic innervation from the ventral tegmental area (VTA; Lewis et al., 1981, Bottjer, 1993, Gale et al., 2008) which neurons display physiological properties very similar to those of mammalian SNc and VTA (Gale and Perkel, 2006). This specialized BG–thalamo-cortical loop devoted to a complex, social, naturally learned sensorimotor task is particularly tractable for elucidating the interwoven sensory, motor and reward signals carried by DA in the BG, and the function of DA signaling in skill learning and execution (Doupe et al., 2005).

Section snippets

DA receptors in the song system

The two families of dopaminergic receptors (D1-like and D2-like) are present in the songbird brain, with particularly high expression levels in the striatum (Casto and Ball, 1994). Interestingly, the song-related BG nucleus Area X displays even higher concentration of both DA receptor subtypes compared to the surrounding striatum (Casto and Ball, 1994). D1-like DA receptor are present to a lesser extent in cortical song-related nuclei such as HVC (used as a proper name), the robust nucleus of

DA effects on mammalian striatal neurons

DA is known for several decades to modulate synaptic transmission and neuronal activity in the mammalian BG (Akaike et al., 1987, Hu and Wang, 1988, Nicola et al., 2000). Because the striatum receives the largest DA input in the BG in mammals (Smith and Villalba, 2008), previous physiological studies have concentrated on DA effects on striatal neurons. While early studies reported an excitatory effect of DA through D1-like receptor and inhibitory through D2 (Albin et al., 1989), the complex

A loop circuit between striatal and dopaminergic neurons in birds

In birds as in mammals, the BG and in particular song-related nucleus Area X receive dense dopaminergic input from the SNc and the VTA (Lewis et al., 1981, Bottjer, 1993, Soha et al., 1996). At first glance, Area X differs from mammalian BG in its gross anatomical structure. Spiny neurons in Area X, homologue to the mammalian striatal projection neurons, do not project outside of Area X (Farries and Perkel, 2002). Instead, Area X contains a class of neurons homologous to mammalian pallidal

Social context-modulated song-related DA signals in the avian BG

Several experimental findings show that the dopaminergic signal is important for song modulation by social context. First, interfering with DA transmission induces profound decrements in female-directed singing and in copulatory behavior in males (Harding, 2004), and on female responses to male songs (Riters and Pawlisch, 2007, Pawlisch and Riters, 2010). Secondly, immunocytochemical data suggest that DA in both song control and motivation brain regions is tightly linked to the regulation of

D1-receptor mediate social context modulation of song spectral variability

As stated in the introduction, adult birds modulate trial-to-trial variability in their song with social context (Sossinka and Böhner, 1980, Kao et al., 2005). More precisely, the spectral variability of syllables from rendition to rendition, measured as the spread of the distribution of fundamental frequencies of syllables displaying a harmonic spectral structure, is decreased when the bird sings courtship songs directed toward a female (Kao et al., 2005, Kao and Brainard, 2006). The output

Tonic DA in the BG regulates exploration–exploitation trade-off

The song-related BG–thalamo-cortical loop is critical for song learning in juvenile birds (Scharff and Nottebohm, 1991), and for song plasticity in adults (Brainard and Doupe, 2000, Andalman and Fee, 2009). Based on all the experimental data summarized in this review, can we postulate a function for the role of the tonic dopaminergic signal in the song-related BG nucleus Area X in song learning? And can this role be extrapolated to other vertebrates?

What about D2?

While I have proposed a function for the tonic dopaminergic signaling through D1-like receptors in the BG of songbird, acting to regulate the balance between exploration and exploitation depending on social context, the function of tonic signaling through D2-like receptors, also present in large amounts in the songbird BG, remains to be elucidated. Although it has been proposed that two different populations of MSN neurons in Area X would express D1-like and D2-like dopaminergic receptors (

Acknowledgements

I am grateful to David J. Perkel, Thomas Boraud and David Hansel for their valuable comments on this manuscript, and to Samuel D Gale for fruitful discussions. This work was supported by the French Agency for Research, Grant ANR-10-PDOC-016-01 and by the European Union, Grant PIRG06-GA-2009-256488.

References (143)

  • S.A. Heimovics et al.

    Evidence that dopamine within motivation and song control brain regions regulates birdsong context-dependently

    Physiol. Behav.

    (2008)
  • M. Joshua et al.

    The dynamics of dopamine in control of motor behavior

    Curr. Opin. Neurobiol.

    (2009)
  • A.C. Kreitzer et al.

    Striatal plasticity and basal ganglia circuit function

    Neuron

    (2008)
  • T.N. Lerner et al.

    Neuromodulatory control of striatal plasticity and behavior

    Curr. Opin. Neurobiol.

    (2011)
  • B.A. Pawlisch et al.

    Selective behavioral responses to male song are affected by the dopamine agonist GBR-12909 in female European starlings (Sturnus vulgaris)

    Brain Res.

    (2010)
  • A.L. Person et al.

    Unitary IPSPs drive precise thalamic spiking in a circuit required for learning

    Neuron

    (2005)
  • L. Petrinovich et al.

    Anim. Behav.

    (1987)
  • O. Pintér et al.

    Seasonal changes in courtship behavior, plasma androgen levels and in hypothalamic aromatase immunoreactivity in male free-living European starlings (Sturnus vulgaris)

    Gen. Comp. Endocrinol.

    (2011)
  • A. Reiner et al.

    Structural and functional evolution of the basal ganglia in vertebrates

    Brain Res. Brain Res. Rev.

    (1998)
  • S.J. Alger et al.

    Lesions to the medial preoptic nucleus differentially affect singing and nest box-directed behaviors within and outside of the breeding season in European starlings (Sturnus vulgaris)

    Behav. Neurosci.

    (2006)
  • V. Aliane et al.

    Cocaine-induced stereotypy is linked to an imbalance between the medial prefrontal and sensorimotor circuits of the basal ganglia

    Eur. J. Neurosci.

    (2009)
  • A.S. Andalman et al.

    A basal ganglia–forebrain circuit in the songbird biases motor output to avoid vocal errors

    Proc. Natl. Acad. Sci. USA

    (2009)
  • B.J. Aragona et al.

    Dopamine regulation of social choice in a monogamous rodent species

    Front. Behav. Neurosci.

    (2009)
  • A. Barbeau

    The “pink spot”, 3,4-dimethoxyphenylethylamine and dopamine. Relationship to Parkinson’s disease and to schizophrenia

    Rev. Can. Biol.

    (1967)
  • Berridge

    The debate over dopamine’s role in reward: the case for incentive salience

    Psychopharmacology

    (2007)
  • S.W. Bottjer

    The distribution of tyrosine hydroxylase immunoreactivity in the brains of male and female zebra finches

    J. Neurobiol.

    (1993)
  • S.W. Bottjer et al.

    Parallel pathways for vocal learning in basal ganglia of songbirds

    Nat. Neurosci.

    (2010)
  • S.W. Bottjer et al.

    Circuits, hormones, and learning: vocal behavior in songbirds

    J. Neurobiol.

    (1997)
  • M.S. Brainard et al.

    Interruption of a basal ganglia-forebrain circuit prevents plasticity of learned vocalizations

    Nature

    (2000)
  • X. Caldú et al.

    Hormonal and genetic influences on processing reward and social information

    Ann. N. Y. Acad. Sci.

    (2007)
  • J.J. Canales et al.

    A measure of striatal function predicts motor stereotypy

    Nat. Neurosci.

    (2000)
  • G.D. Carrillo et al.

    Is the songbird Area X striatal, pallidal, or both? An anatomical study

    J. Comp. Neurol.

    (2004)
  • J.M. Casto et al.

    Characterization and localization of D1 dopamine receptors in the sexually dimorphic vocal control nucleus, Area X, and the basal ganglia of European starlings

    J. Neurobiol.

    (1994)
  • J.J. Clark et al.

    Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals

    Nat. Methods

    (2010)
  • J.Y. Cohen et al.

    Neuron-type-specific signals for reward and punishment in the ventral tegmental area

    Nature

    (2012)
  • R.M. Costa

    Plastic corticostriatal circuits for action learning: what’s dopamine got to do with it?

    Ann. N. Y. Acad. Sci.

    (2007)
  • L. Ding et al.

    Dopamine modulates excitability of spiny neurons in the avian basal ganglia

    J. Neurosci. Off. J. Soc. Neurosci.

    (2002)
  • L. Ding et al.

    Long-term potentiation in an avian basal ganglia nucleus essential for vocal learning

    J. Neurosci. Off. J. Soc. Neurosci.

    (2004)
  • L. Ding et al.

    Presynaptic depression of glutamatergic synaptic transmission by D1-like dopamine receptor activation in the avian basal ganglia

    J. Neurosci. Off. J. Soc. Neurosci.

    (2003)
  • A.J. Doupe et al.

    Birdsong and human speech: common themes and mechanisms

    Annu. Rev. Neurosci.

    (1999)
  • M.a. Farries et al.

    A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus

    J. Neurosci. Off. J. Soc. Neurosci.

    (2002)
  • M.S. Fee et al.

    A hypothesis for basal ganglia-dependent reinforcement learning in the songbird

    Neuroscience

    (2011)
  • H.L. Fields et al.

    Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement

    Annu. Rev. Neurosci.

    (2007)
  • E. Fino et al.

    Spike-timing dependent plasticity in the striatum

    Front. Synap. Neurosci.

    (2010)
  • C.D. Fiorillo et al.

    Discrete coding of reward probability and uncertainty by dopamine neurons

    Science

    (2003)
  • S.B. Floresco et al.

    Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission

    Nat. Neurosci.

    (2003)
  • M.J. Frank et al.

    Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation

    Nat. Neurosci.

    (2009)
  • S.D. Gale et al.

    Properties of dopamine release and uptake in the songbird basal ganglia

    J. Neurophysiol.

    (2005)
  • S.D. Gale et al.

    Physiological properties of zebra finch ventral tegmental area and substantia nigra pars compacta neurons

    J. Neurophysiol.

    (2006)
  • S.D. Gale et al.

    A basal ganglia pathway drives selective auditory responses in songbird dopaminergic neurons via disinhibition

    J. Neurosci. Off. J. Soc. Neurosci.

    (2010)
  • Cited by (12)

    • Capturing the Effects of Domestication on Vocal Learning Complexity

      2021, Trends in Cognitive Sciences
      Citation Excerpt :

      In male zebra finches, neuronal firing in the primarily dopaminergic midbrain VTA, which projects to Area X (see Figure I in Box 1), correlates with the singing of a more stereotyped female-directed song [57]. Social-context–dependent alterations to dopaminergic signaling regulate the change from juvenile exploratory vocalizations to the adult crystallization of a more stereotyped song [58]. Independently of the influences of social context on song learning, stimulation or inhibition of midbrain (VTA) dopaminergic projections to the songbird striatum, driven by auditory feedback, is sufficient to guide syllable pitch learning [59].

    • Social context-dependent singing alters molecular markers of synaptic plasticity signaling in finch basal ganglia Area X

      2021, Behavioural Brain Research
      Citation Excerpt :

      Akt protein levels trended downward with more time spent singing UD song (Fig. 8C), but there was no relationship between the number of UD motifs to Akt protein levels (Fig. 8D). UD song has more acoustic variability from rendition-to-rendition compared to FD song [32,33]. These song output differences are supported by studies measuring neural activity of neurons in Area X.

    • Dopamine depletion affects communicative intentionality in Parkinson's disease patients: Evidence from action kinematics

      2016, Cortex
      Citation Excerpt :

      As a final point, the present findings on the possible relationship between dopamine systems and social behaviour are also in line with animal studies showing that in rats D2-R-related functions balance pro- and antisocial behaviours (Aragona et al., 2006; Couppis, Kennedy, & Stanwood, 2008). In songbirds, individuals which show communicative ‘disorders’ can be motivated to communicate through pharmacological manipulations of dopamine receptors, suggesting dopamine as a candidate neurotransmitter system involved in stimulating social communication (Leblois, 2013). In conclusion, our results support the view that specific kinematic patterns characterize and distinguish communicative actions from actions executed with a purely individual intent.

    • Social intentions in Parkinson's disease patients: A kinematic study

      2015, Cortex
      Citation Excerpt :

      The fact that the VS may be activated during social interaction implies that the basal ganglia (BG), the main recipient of VS outputs (Draganski et al., 2008), are involved in these endeavours. The BG are implicated in sensorimotor learning and receive a strong dopaminergic signal, which has been shown to play an important role in social interactions (Leblois, 2013). Despite this evidence, however, how the BG works to integrate social cues and how a dysfunction of the dopaminergic system can affect the ability to plan and to execute actions in a social context remains largely unexplored.

    • A neural network model for timing control with reinforcement

      2022, Frontiers in Computational Neuroscience
    View all citing articles on Scopus
    View full text