The [3H]resiniferatoxin (RTX) binding assay using membrane preparations has been used to identify and characterize the vanilloid receptors in the central and peripheral nervous system of different species. In the present study, using cultured adult rat dorsal root ganglion neurons either in suspension or attached to the tissue culture plates, we developed an assay to measure specific [3H]RTX binding by the intact cells. We were able to characterize the vanilloid binding characteristics of the neurons and compared those to the properties of vanilloid binding sites present in rat dorsal root ganglia membrane preparations. We found that [3H]RTX bound with similar affinity and positive cooperativity to attached neurons (cultured for 5 days before being assayed), neurons in suspension (using a filtration assay) and dorsal root ganglion membrane preparations. Dissociation constants obtained in the three assays were 47.6 +/- 3.5 pM, 38.4 +/- 3.1 pM and 42.6 +/- 3.1 pM, respectively. The cooperativity indexes determined by fitting the data to the Hill equation were 1.73 +/- 0.11, 1.78 +/- 0.12 and 1.78 +/- 0.09, respectively. The maximal binding capacity was 0.218 +/- 0.026 fmol/10(3) cells and 0.196 +/- 0.021 fmol/10(3) cells in the case of the attached cells and cells in suspension, respectively. Nonradioactive RTX, capsaicin, capsazepine and resiniferonol 20-homovanillylamide fully displaced specifically bound [3H]RTX from cells in suspension with Ki and Hill coefficient values of 42.5 +/- 5.3 pM, 2.06 +/- 0.16 microM, 3.16 +/- 0.21 microM and 32.4 +/- 4.1 nM and 1.79 +/- 0.17, 1.68 +/- 0.06, 1.72 +/- 0.11 and 1.81 +/- 0.12, respectively. Structure-activity analysis of different vanilloid derivatives revealed that the various compounds have distinct potencies for receptor binding and inducing 45Ca uptake in rat dorsal root ganglion neurons. Affinities for receptor binding and stimulation of 45Ca uptake of RTX, resiniferonol 20-homovanillylamide, RTX-thiourea, tinyatoxin, phorbol 12,13-dibenzoate 20-homovanillylamide and capsaicin were 38.5 +/- 2.9 pM, 25.7 +/- 3.0 nM, 68.5 +/- 3.8 nM, 173 +/- 25 pM, 7.98 +/- 0.83 microM and 4.93 +/- 0.35 microM as compared to 0.94 +/- 0.12 nM, 26.5 +/- 3.5 nM, 149 +/- 30 nM, 1.46 +/- 0.25 nM, 1.41 +/- 0.48 microM and 340 +/- 57 nM. Computer fitting of the data yielded Hill coefficient values indicating positive cooperativity of receptor binding; however, stimulation of 45Ca uptake appeared to follow a non-cooperative mechanism of action. The competitive capsaicin antagonist capsazepine inhibited specific binding of [3H]RTX by rat dorsal root ganglion membrane preparations with Ki and Hill coefficient values of 3.89 +/- 0.38 microM and 1.74 +/- 0.11. On the other hand it inhibited the induction of 45Ca uptake into the cells induced by capsaicin and RTX in a non-cooperative fashion with Ki values of 271 +/- 29 nM and 325 +/- 47 nM. Our results show that the membrane binding assay relates to the reality of receptor function in the intact, cultured neurons, both in terms of affinity and positive cooperativity. However the different vanilloid derivatives displayed markedly distinct structure-activity relations for high affinity receptor binding and stimulation of 45Ca uptake into rat dorsal root ganglion neurons. Among various explanations for this discrepancy, we favor the possibility that the two assays detect distinct classes of the vanilloid (capsaicin) receptor present in primary sensory neurons.