Serotonin–dopamine interaction: electrophysiological evidence

https://doi.org/10.1016/S0079-6123(08)00903-5Get rights and content

Abstract

In this review, the most relevant data regarding serotonin (5-hydroxytryptamine, 5-HT)/dopamine (DA) interaction in the brain, as studied by both in vivo and in vitro electrophysiological methods, are reported and discussed. The bulk of neuroanatomical data available clearly indicate that DA-containing neurons in the brain receive a prominent innervation from 5-HT originating in the raphe nuclei of the brainstem. Furthermore, this modulation seems to be reciprocal; DA neurons innervate the raphe nuclei and exert a tonic excitatory effect on them. Compelling electrophysiological data show that 5-HT can exert complex effects on the electrical activity of midbrain DA neurons mediated by the various receptor subtypes. The main control seems to be inhibitory, this effect being more marked in the ventral tegmental area (VTA) as compared to the substantia nigra pars compacta (SNc). In spite of a direct effect of 5-HT by its receptors located on DA cells, 5-HT can modulate their activity indirectly, modifying γ-amino-n-butyric acid (GABA)-ergic and glutamatergic input to the VTA and SNc. Although 5-HT/DA interaction in the brain has been extensively studied, much work remains to be done to clarify this issue. The recent development of subtype-selective ligands for 5-HT receptors will not only allow a detailed understanding of this interaction but also lead to development of new treatment strategies, appropriate for those neuropsychiatric disorders in which an alteration of the 5-HT/DA balance is supposed.

Introduction

This chapter is devoted to the analysis and discussion of the most relevant data regarding serotonin (5-hydroxytryptamine, 5-HT)/dopamine (DA) interaction in the brain, as studied by electrophysiological methods. Both in vivo and in vitro findings will be reviewed, although emphasis will be given to data obtained by in vivo extracellular single-cell recordings. The bulk of neuroanatomical data available clearly indicate that DA-containing neurons in the brain receive a prominent innervation from 5-HT originating in the raphe nuclei of the brainstem (Fig. 1). Thus, the detailed knowledge of this neuroanatomical wiring is an obvious prerequisite for the understanding of the functional interaction of 5-HT and DA systems, including the electrophysiological findings.

The presence of 5-HT-containing nerve fibres in the substantia nigra (SN) and the ventral tegmental area (VTA) was first demonstrated by Fuxe (1965) using formaldehyde-induced fluorescence histochemistry. Subsequent studies have confirmed and extended these findings, showing also that 5-HT fibres coming from the dorsal raphe nucleus (DRN) were distributed significantly to the SN and the VTA (Fibiger and Miller, 1977; Azmitia and Segal, 1978; Steinbusch, 1981; Hervé et al., 1987; Mori et al., 1987; Wirtshafter et al., 1987; Lavoie and Parent, 1990; Van Bockstaele et al., 1994; Van Bockstaele and Pickel, 1994; Moukhles et al., 1997; Vertes and Linley, 2007), and to the main projection areas of the nigro-striatal and the mesolimbic DA-ergic system, such as the corpus striatum, the globus pallidus, the nucleus accumbens, amygdala, olfactory tubercle and the frontal cortex (Azmitia and Segal, 1978; Steinbusch, 1981; Lavoie and Parent, 1990; Vertes and Linley, 2007). In contrast, 5-HT fibres arising from medial raphe nucleus (MRN) innervate the VTA but not the SN, although the concentration of 5-HT terminals in this area is less dense as compared to the SN (Fibiger and Miller, 1977; Azmitia and Segal, 1978; Steinbusch, 1981; Mori et al., 1987; Wirtshafter et al., 1987; Lavoie and Parent, 1990; Vertes and Linley, 2007). In fact, it is noteworthy that the SN receives the densest 5-HT innervation of the brain in several animal species, including rats (Palkovits et al., 1974; Saavedra, 1977; Moukhles et al., 1997), monkeys (Shannak and Ornykiewicz, 1980; Lavoie and Parent, 1990) and humans (Fahn et al., 1971; Mackay et al., 1978). Other investigations revealed the raphe-nigral projection arises mainly from the DRN and terminates principally in the substantia nigra pars reticulata (SNr) (Azmitia and Segal, 1978; Mori et al., 1987; Lavoie and Parent, 1990; Corvaja et al., 1993; Moukhles et al., 1997). Thus, it has been reported that 5-HT neurons innervate both DA and non-DA neurons in the VTA (Hervé et al., 1987; Van Bockstaele et al., 1994), with as many as 50% of [3H]5-HT forming synaptic contacts in this area (Hervé et al., 1987). Of these 5-HT terminals, about 32% formed synaptic junctions with perikarya or dendrites containing tyrosine hydroxylase (TH) immunoreactivity (i.e. DA neurons) (Van Bockstaele et al., 1994).

Quantitative autoradiographic mapping of 5-HT in the rat brain showed the presence of 5-HT receptor subtypes on both nigro-striatal and mesocorticolimbic DA-ergic systems (Pazos et al., 1985; Pazos and Palacios, 1985). Thus, by using tritiated 5-HT ligands, it was found that 5-HT1 receptors were present in the SN and, in particular, in the SNr (which contains high concentrations of 5-HT1B receptors), the corpus striatum and several areas of the mesocorticolimbic system such as the VTA, the nucleus accumbens, the amygdala, the olfactory tubercle and the frontal cortex (Pazos and Palacios, 1985). Also, 5-HT2A receptors (which at that time were simply named 5-HT2) are distributed in most DA-ergic areas of the brain, including the substantia nigra pars compacta (SNc), the VTA, the striatum, the nucleus accumbens, the amygdala, the olfactory tubercle, the entorhinal cortex and the frontal cortex (Pazos et al., 1985). Autoradiographic mapping studies also showed the presence of 5-HT2C receptors (which at that time were defined 5-HT1C, and were labelled by tritiated 5-HT and mesulergine) in several brain regions, including nuclei of origin and terminal areas of central DA-ergic systems, such as the SNc, the VTA, the striatum, the nucleus accumbens, the entorhinal cortex and the frontal cortex (Pazos and Palacios, 1985). Subsequent autoradiographic studies confirmed the presence of 5-HT1B, 5-HT2A and 5-HT2C receptors in several DA-ergic areas of the human brain (Hoyer et al., 1986; Pazos et al., 1987; Marazziti et al., 1999). In addition, relatively high densities of 5-HT3 and 5-HT4 receptor subtypes have been detected in terminal areas of the mesolimbic DA-ergic system, whereas low or undetectable levels of 5-HT3 receptors are present in the VTA, or the nigro-striatal DA system (Barnes et al., 1990; Kilpatrick et al., 1996) (Table 1).

A number of studies, using in situ hybridization histochemistry and immunohistochemical techniques, have shown the presence of 5-HT2A and 5-HT2C mRNAs receptor proteins localized on neuronal components of the nigro-striatal and mesocorticolimbic DA-ergic systems of rodents (Molineaux et al., 1989; Mengod et al., 1990a, Mengod et al., 1990b; Pompeiano et al., 1994; Wright et al., 1995; Ward and Dorsa, 1996; Eberle-Wang et al., 1997; Clemett et al., 2000; Doherty and Pickel, 2000; Ikemoto et al., 2000; Bubar and Cunningham, 2007; Liu et al., 2007) and humans (Abramowski et al., 1995; Pasqualetti et al., 1999), whereas 5-H2B receptor proteins were undetectable on these neurons (Duxon et al., 1997). It is important to note that double-label in situ hybridization in the SN reveals co-expression of 5-HT2C receptor mRNA with glutamic acid decarboxylase (GAD) but not with TH mRNA, indicating that 5-HT2C receptors are restricted to γ-amino-n-butyric acid (GABA)-ergic neurons, both in adult rats (Eberle-Wang et al., 1997) and in humans (Pasqualetti et al., 1999). However, it was recently found that in the VTA, 5-HT2C receptor protein is expressed on both DA and GABA-containing neurons (Bubar and Cunningham, 2007). Similarly, 5-HT2A receptors are localized on both DA (about 20–40% of the population, according to different authors) and non-DA neurons in the VTA (Cornea-Hebert et al., 1999; Doherty and Pickel, 2000; Ikemoto et al., 2000; Nocjar et al., 2002), although in this case the neurochemical nature of non-DA neurons was not established. The presence of both 5-HT2A and 5-HT2C receptors has also been detected in neurons containing GABA, acetylcholine, enkephalin, substance P or dynorphin, in various terminal regions of the nigro-striatal and mesocorticolimbic DA system, such as the striatum (Ward and Dorsa, 1996), the nucleus accumbens, the lateral amygdala, the pyriform cortex and the medial prefrontal cortex (mPFC) (Morilak et al., 1993; Liu et al., 2007).

The 5-HT2A and 5-HT2C receptors display 50% overall amino acid sequence homology and >80% sequence homology within the transmembrane domain regions (Boess and Martin, 1994; Martin and Humphrey, 1994). Likewise, the 5-HT2A and 5-HT2C receptors are both coupled to Gαq/11 and activate the downstream effector phospholipase C (PLC), thus stimulating the hydrolysis of membrane phospholipids and the production of inositol-1-4,5-triphosphate and diacylglycerol, leading to increased intracellular Ca2+ (Hoyer et al., 2002). Moreover, both receptors couple with Gαq/11 to activate phospholipase A2 (PLA2), independent of PLC activation (Felder et al., 1990). Despite these similarities in signalling mechanisms, subtle differences between 5-HT2A and 5-HT2C signalling cascades also exist (Table 2). In addition, agonists have been shown to activate signalling pathways differentially and to couple to their effectors in an agonist-independent manner (constitutive activity), although this differs with regard to receptor and effectors (Berg et al., 2005). For example, the 5-HT2C receptors express greater constitutive activity towards PLC, but the 5-HT2C receptor has relatively weak constitutive activity towards PLA2 (determined by arachidonic acid release) (Berg et al., 2005).

Section snippets

Dopamine modulation of serotonergic neurons activity

It is noteworthy that midbrain DA nuclei also control the activity of 5-HT neurons; thus, the control is reciprocal in nature. Indeed, a dense direct projection from the SNc to the DRN by retrograde tracer technique has been characterized (Sakai et al., 1977). These results have been extended successively using injection of tritiated leucine and proline into the VTA and SNc-labelled fibres, confirming an overlapping descending pathway to the DRN (Beckstead et al., 1979). The DA-ergic

Conclusions and future directions

Despite the fact that electrophysiological data regarding the interaction between the serotonergic and the DA-ergic systems in the brain have accumulated over a period of about 35 years of intensive research, the exact effect of 5-HT on the activity of DA neurons is far from being completely understood.

The modulation of DA-ergic activity by the serotonergic system is complex, and complicated by the large number of 5-HT receptor subtypes present in DA nuclei and in their target areas. It can

Abbreviations

    Ψ

    functional operator equivalent to the density power spectrum of the signals

    5,7-DHT

    5,7-dihydroxytryptamine

    5-HT

    5-hydroxytryptamine, serotonin

    CNS

    central nervous system

    DA

    dopamine

    DRN

    dorsal raphe nucleus

    EPSC

    excitatory postsynaptic current

    GABA

    γ-amino-n-butyric acid

    GAD

    glutamic acid decarboxylase

    GIRKs

    inwardly rectifying potassium channels

    GLU

    glutamate

    Ih

    hyperpolarization-activated cation current

    IPSP

    inhibitory postsynaptic potential

    IS-SD

    initial segment-somatodendritic

    mGLUR

    metabotropic glutamate receptor

References (164)

  • M.J. Bubar et al.

    Distribution of serotonin 5-HT2C receptors in the ventral tegmental area

    Neuroscience

    (2007)
  • D.L. Cameron et al.

    A subset of ventral tegmental area neurons is inhibited by dopamine, 5-hydroxytryptamine and opioids

    Neuroscience

    (1997)
  • L.A. Chiodo

    Dopamine-containing neurons in the mammalian central nervous system: electrophysiology and pharmacology

    Neurosci. Biobehav. Rev.

    (1988)
  • D.A. Clemett et al.

    Immunohistochemical localisation of the 5-HT2C receptor protein in the rat CNS

    Neuropharmacology

    (2000)
  • N. Corvaja et al.

    Ultrastructure and synaptic targets of the raphe-nigral projection in the rat

    Neuroscience

    (1993)
  • G. Di Giovanni et al.

    Selective blockade of serotonin2C/2B receptors enhances mesolimbic and mesostriatal DA-ergic function: a combined in vivo electrophysiological and microdialysis study

    Neuroscience

    (1999)
  • G. Di Giovanni et al.

    m-Chlorophenylpiperazine excites non-DA-ergic neurons in the rat substantia nigra and ventral tegmental area by activating serotonin-2C receptors

    Neuroscience

    (2001)
  • V. Di Matteo et al.

    Role of 5-HT2C receptors in the control of central dopamine function

    Trends Pharmacol. Sci.

    (2001)
  • V. Di Matteo et al.

    SB 242084, a selective serotonin2C receptor antagonist, increases DA-ergic transmission in the mesolimbic system

    Neuropharmacology

    (1999)
  • V. Di Matteo et al.

    Biochemical and electrophysiological evidence that RO 60-0175 inhibits mesolimbic DA-ergic function through serotonin2C receptors

    Brain Res.

    (2000)
  • M.D. Doherty et al.

    Ultrastructural localization of the serotonin 2A receptor in DA-ergic neurons in the ventral tegmental area

    Brain Res.

    (2000)
  • A. Dray et al.

    The dorsal and medial raphe projections to the substantia nigra in the rat: electrophysiological, biochemical and behavioural observations

    Brain Res.

    (1978)
  • A. Dray et al.

    Evidence for the existence of a raphe projection to the substantia nigra in the rat

    Brain Res.

    (1976)
  • M.S. Duxon et al.

    Evidence for expression of the 5-hydroxytryptamine-2B receptor protein in the rat central nervous system

    Neuroscience

    (1997)
  • S. Fahn et al.

    Monoamines in the human neostriatum; topographic distribution in normals and in Parkinson's disease and their role in akinesia, rigidity, chorea and tremor

    J. Neurol. Sci.

    (1971)
  • H.C. Fibiger et al.

    An anatomical and electrophysiological investigation of the serotonergic projection from the dorsal raphe nucleus to the substantia nigra in the rat

    Neuroscience

    (1977)
  • A.A. Grace et al.

    Intracellular and extracellular electrophysiology of nigral DA-ergic neurons — 1. Identification and characterization

    Neuroscience

    (1983)
  • A.A. Grace et al.

    Intracellular and extracellular electrophysiology of nigral DA-ergic neurons — 2. Action potential generating mechanisms and morphological correlates

    Neuroscience

    (1983)
  • D. Hervé et al.

    Serotonin axon terminals in the ventral tegmental area and synaptic input to DA-ergic neurons

    Brain Res.

    (1987)
  • D. Hoyer et al.

    Molecular, pharmacological and functional diversity of 5-HT receptors

    Pharmacol. Biochem. Behav.

    (2002)
  • D. Hoyer et al.

    Serotonin receptors in the human brain. II. Characterization and autoradiographic localization of 5-HT1C and 5-HT2 recognition sites

    Brain Res.

    (1986)
  • K. Ikemoto et al.

    Human midbrain dopamine neurons express serotonin 2A receptor: an immunohistochemical demonstration

    Brain Res.

    (2000)
  • R.W. Invernizzi et al.

    Selective activation of 5-HT2C receptors stimulated GABA-ergic function in the rat substantia nigra pars reticulata: a combined in vivo electrophysiological and neurochemical study

    Neuroscience

    (2007)
  • M.D. Kelland et al.

    Ascending afferent regulation of rat midbrain dopamine neurons

    Brain Res. Bull.

    (1993)
  • K. Kitahama et al.

    Distribution of dopamine-immunoreactive fibers in the rat brainstem

    J. Chem. Neuroanat.

    (2000)
  • S. Liu et al.

    Serotonin2C receptor localization in GABA neurons of the rat medial prefrontal cortex: implications for understanding the neurobiology of addiction

    Neuroscience

    (2007)
  • D. Marazziti et al.

    Distribution and characterization of [3H]mesulergine binding in the human brain postmortem

    Eur. Neuropsychopharmacol.

    (1999)
  • G.R. Martin et al.

    Receptors for 5-hydroxytryptamine: current perspectives on classification and nomenclature

    Neuropharmacology

    (1994)
  • G.K. Aghajanian et al.

    DA-ergic and nonDA-ergic neurons of the substantia nigra: differential responses to putative transmitters

  • G.K. Aghajanian et al.

    Dopamine “autoreceptors”: pharmacological characterization by microiontophoretic single cell recording studies

    Naunyn Schmiedebergs Arch. Pharmacol.

    (1977)
  • T.K. Aman et al.

    D2-like dopamine receptors depolarize dorsal raphe serotonin neurons through the activation of nonselective cationic conductance

    J. Pharmacol. Exp. Ther.

    (2007)
  • J.L. Andersson et al.

    Ritanserin potentiates the stimulatory effects of raclopride on neuronal activity and dopamine release selectively in the mesolimbic DA-ergic system

    Naunyn Schmiedebergs Arch. Pharmacol.

    (1995)
  • L. Arborelius et al.

    The 5-HT1A selective ligands, (R)-8-OHDPAT and (S)-UH-301, differentially affect the activity of midbrain dopamine neurons

    Naunyn Schmiedebergs Arch. Pharmacol.

    (1993)
  • E.C. Azmitia et al.

    An autoradiographic analysis of the differential ascending projections of the dorsal and median raphe nuclei in the rat

    J. Comp. Neurol.

    (1978)
  • T.P. Blackburn et al.

    Effect of acute and chronic administration of the selective 5-HT2C receptor antagonist SB-243213 on midbrain dopamine neurons in the rat: an in vivo extracellular single cell study

    Synapse

    (2002)
  • P. Blier et al.

    Effect of repeated amiflamine administration on serotonergic and noradrenergic neurotransmission: electrophysiological studies in the rat CNS

    Naunyn Schmiedebergs Arch. Pharmacol.

    (1986)
  • M.S. Brodie et al.

    Serotonin potentiates ethanol-induced excitation of ventral tegmental area neurons in brain slices from three different rat strains

    J. Pharmacol. Exp. Ther.

    (1995)
  • S.M. Brodie et al.

    Serotonin potentiates dopamine inhibition of ventral tegmental area neurons in vitro

    J. Neurophysiol.

    (1996)
  • M.J. Bubar et al.

    Serotonin 5-HT2A and 5-HT2C receptors as potential targets for modulation of psychostimulants use and dependence

    Curr. Top. Med. Chem.

    (2006)
  • B.S. Bunney et al.

    Comparison of effects of L-Dopa, amphetamine and apomorphine on firing rate of rat DA-ergic neurones

    Nat. New Biol.

    (1973)
  • Cited by (0)

    View full text