Intracellular recordings were made with conventional microelectrodes and with whole-cell patch-clamp electrodes from neurons of the rat ventral tegmental area and substantia nigra zona compacta in vitro. Neurons were distinguished as principal cells and secondary cells; it is known from previous work that most principal cells contain dopamine whereas secondary cells do not. 5-Hydroxytryptamine (5-HT; 3-100 microM) depolarized (or evoked an inward current at -60 mV) 46% of 153 principal cells; a small proportion (11%) of cells were hyperpolarized (or showed outward current at -60 mV). Secondary cells were equally likely to be depolarized (or inward current at -60 mV, 30% of 80 cells) or hyperpolarized (or outward current at -60 mV, 28%). approximately 40% of each type of cell were unaffected by 5-HT. Depolarizing responses of 5-HT were mimicked by (+/-)-1-(2,5-dimethoxy-4-iodophenyl)- 2-aminopropane (DOI) and blocked by ketanserin. Hyperpolarizing responses were mimicked by dipropyl-5-carboxamidotryptamine and reversed polarity at the K+ equilibrium potential. Inhibitory postsynaptic potentials (or currents) mediated at GABAA receptors occurred spontaneously in some principal cells; they were reversibly blocked by tetrodotoxin and bicuculline. 5-HT either increased or decreased the frequency of these synaptic potentials but did not change their mean amplitude or decay time.(ABSTRACT TRUNCATED AT 250 WORDS)