1. We investigated the effects of serotonin (5-hydroxytryptamine, 5-HT) on whole-cell barium currents through calcium channels in visualized neonatal rat hypoglossal motoneurones (HMs) in a thin brainstem slice preparation. 2. High voltage-activated (HVA) currents were elicited by depolarizing voltage steps from -70 to 0 mV; low voltage-activated (LVA) currents were evoked using steps to between -30 and -40 mV from hyperpolarized potentials (< -80 mV). 5-HT (1.0 microM) inhibited HVA currents by at least 10% in 70% of HMs tested (n = 99); in those responsive neurones, 5-HT decreased HVA current by 22 +/- 1.3% (mean +/- S.E.M.). In contrast, 5-HT had no effect on LVA current amplitude in HMs (n = 7). 3. Calcium current inhibition was mimicked by 5-carboxamidotryptamine maleate (5-CT), a 5-HT1 receptor agonist, and by R(+)-8-hydroxydipropylaminotetralin hydrobromide (8-OH-DPAT), a specific 5-HT1A agonist; N-(3-trifluoromethylphenyl) piperazine hydrochloride (TFMPP), a 5-HT1B agonist, was without effect. The effect of 5-HT was blocked by the 5-HT1A antagonist 1-(2-methoxyphenyl)-4-[4-(2-phthalimido)butyl]piperazine hydrobromide (NAN-190) but not by ketanserin, a 5-HT2A/2C antagonist. Although R(-)-1-(2,5-dimethoxy-4-iodophenyl)-2-aminopropane hydrochloride (DOI), a 5-HT2A/2C agonist, mimicked the current inhibition by 5-HT, it was ineffective in the presence of NAN-190. These data indicate that 5-HT1A receptors mediate calcium current inhibition by 5-HT. 4. Following application of either omega-conotoxin-GVIA (omega-CgTX) or omega-agatoxin-IVA (omega-Aga-IVA), to block N- and P-type components of calcium current, the 5-HT-sensitive current was reduced; 5-HT had no effect on the current remaining after application of both toxins. Thus, 5-HT inhibits both N- and P-type calcium currents in neonatal HMs. 5. Inhibition of HVA current by 5-HT was irreversible, and subsequent applications of 5-HT were occluded, when GTP gamma S was substituted for GTP in the pipette. In addition, inhibition of HVA current by 5-HT was relieved following depolarizing prepulses. These data indicate that inhibition of calcium channels by 5-HT is mediated by G proteins. 6. Under current clamp, both 5-HT and 8-OH-DPAT decreased the amplitude of the after-hyperpolarization (AHP) that followed action potentials, indicating involvement of a 5-HT1A receptor.(ABSTRACT TRUNCATED AT 400 WORDS)