Serotonin Receptor Subtypes and Ligands

Richard A. Glennon, Malgorzata Dukat and Richard B. Westkaemper

The neurotransmitter serotonin (5-hydroxytryptamine, 5-HT) mediates a wide range of physiological functions by interacting with multiple receptors, and these receptors have been implicated as playing important roles in certain pathological and psychopathological conditions. In the past 16 years, seven distinct families of 5-HT receptors have been identified (5-HT1–5-HT7), and subpopulations have been described for several of these. At least 15 subpopulations now have been cloned. The profusion of 5-HT receptors should eventually allow a better understanding of the different and complex processes in which 5-HT is involved. Antithetically, this ever expanding list of 5-HT receptors has made it rather difficult to unravel the role of 5-HT receptor subpopulations due to the lack of suitably selective agents. Additionally, it has become a nightmare for those involved in identifying or developing site-selective agonists and antagonists. The present review describes the different populations and nomenclature of 5-HT receptors, their second messenger systems, their psychopharmacological relevance, and agents used to investigate the receptors. Because the literature on serotonin is quite expansive (a MEDLINE search reveals that >15,000 papers have been published since 1990) and because this review represents an update of an earlier review (51), the primary focus will be on the more recent literature; some references used previously (particularly those appearing before 1990) will not be repeated here but can be found in the original review (51). Other helpful review articles covering specific topics are also available: anatomy and cell biology of serotonergic systems (11), receptor distribution (66,127), binding profiles of serotonergic ligands (66,158), commonly used radioligands (66), electrophysiology (5), molecular biology (17,109,127, 135), site-directed mutagenesis (22), molecular graphics modeling of 5-HT receptors (152), 5-HT receptor evolution (110,148), 5-HT receptor nomenclature (66,86), signal transduction pathways (126), and functional/clinical significance (128).

Serotonin was discovered in the late 1940s, and within a decade there were indications for its existence in the central nervous system of animals and for a neurotransmitter function (e.g., 108). By the late 1950s, evidence for 5-HT receptor heterogeneity was found in the periphery, and in 1979 two distinct populations of 5-HT binding sites were identified in rat brain: 5-HT1 and 5-HT2 sites. Soon thereafter, it became apparent that 5-HT1 sites were heterogeneous. Additional sites, distinct from 5-HT1 and 5-HT2 sites, were also identified and termed 5-HT3 receptors. In an attempt to organize the rapidly proliferating nomenclature of 5-HT receptors, specific definitions and criteria were suggested. The different populations of sites were classified as 5-HT1-like, 5-HT2, and 5-HT3. The 5-HT2 receptors and 5-HT3 receptors seemed to correspond to the peripheral 5-HT-D and 5-HT-M receptors, respectively, identified by Gaddum and Picarelli in the late 1950s. Although some of the classification criteria are still applied today, advances in molecular biology, and the discovery of additional sites that seemingly failed to meet the above criteria and definitions resulted in some modifications in nomenclature. For example, 5-HT1-like sites were originally defined, at least in part, as those possessing a high affinity for 5-HT and a structurally-related agent 5-carboxamidotryptamine (5-carbamoyltryptamine; 5-CT). Certain newer 5-HT1 sites are now recognized to bind 5-CT only with low affinity (e.g., 5-HT1E sites); and some non-5-HT1 sites (e.g., 5-HT7) bind both 5-HT and 5-CT with subnanomolar affinities. 5-HT2 sites were originally defined by their low affinity for 5-HT (i.e., Ki » 500–1,200 nM); but it is now known, using the appropriate radioligands (e.g., [125I]DOI, [3H]DOB), that 5-HT binds at 5-HT2 sites with high (i.e., Ki <10 nM) affinity. More recently, it has been shown that most 5-HT1-like receptors share an amino acid sequence homology of at least 50%, display high affinity for 5-HT, and couple to the inhibition of adenylate cyclase activity as a primary coupling pathway (62). However, 5-HT1C receptors possess a 78% sequence homology with 5-HT2 receptors and, like 5-HT2 receptors, are coupled to a phosphoinositol second messenger system. 5-HT1C receptors are now considered to be members of the 5-HT2 family and have been variously termed 5-HT2B, 5-HT2b, and 5-HT2C receptors, relative to the originally defined 5-HT2 receptors (which have been variously referred to as 5-HT2A, 5-HT2a or, simply, 5-HT2 receptors). Adding to the confusion, the terms 5-HT2A and 5-HT2B were also briefly used to describe the high- and low-affinity states of the original 5-HT2 receptors. Today, the term 5-HT2 is used only when referring to the entire family of 5-HT2 receptors, the original 5-HT2 receptors are called 5-HT2A receptors, and 5-HT1C receptors are now referred to as 5-HT2C receptors. In the remainder of this chapter, the new 5-HT receptor nomenclature (66) will be employed. Receptor classification should meet certain criteria: operational (drug-related characteristics), transductional (receptor-effect coupling events), and structural (gene and receptor structural sequences for nucleotide and amino acid components, respectively) (66). Not all populations of 5-HT receptors have been fully characterized. Newly identified recombinant receptors are usually described in lower case letters until operational and transductional data are available (e.g., 5-ht5A, 5-ht5B) [65]; this protocol, although fully acknowledged, will not be used in the present chapter simply for the sake of convenience. The different terminology already in the literature often makes 5-HT receptor nomenclature quite confusing; lists some of the various nomenclatures that have been employed for 5-HT receptors. However, caution must be exercised when reading some of the older primary literature. In addition to the receptor types listed in , several other 5-HT1-like receptors, identified primarily in the periphery, have been described; these (including 5-HT1P receptors) have been referred to as orphan 5-HT receptors (66).

The first two families of 5-HT receptors to be described, 5-HT1 and 5-HT2 (now 5-HT2A) receptors, were the result of radioligand binding studies and were an extension of the discovery that the dopaminergic label [3H]spiperone labeled what appeared to be a population of 5-HT receptors. Since then, several different techniques have been used to initially characterize and define the various populations of 5-HT receptors. Although discussed in greater detail below, for purpose of introduction suffice it to say that radioligand binding (5-HT1A, 5-HT1B, 5-HT1D, 5-HT1E, 5-HT1S, and 5-HT2A), autoradiography and binding (5-HT2C), molecular biology (5-HT1Da, 5-HT1Db, 5-HT1Eb, 5-HT1F, 5-HT2B, 5-HT5A, 5-HT5B, 5-HT6, 5-HT7), and various functional assays (5-HT1P, 5-HT3, 5-HT4) accounted for the discovery of the different populations of sites. Some of the originally described 5-HT receptors, or aspects thereof, already have been the subject of extensive review; where appropriate, reference will be made to review articles.

Numerous agents have been used to investigate 5-HT receptors. Even more bewildering than receptor classification is the selectivity of various 5-HT agonists and antagonists; with the continued identification of new receptor populations, agents once considered selective for a particular population of sites are now realized to be considerably less selective. This has led to the concept of semiselective agents—agents with selectivity for only two or three populations of 5-HT receptors. By a process of elimination, judicious selection of several semiselective agents can often aid in classification of a particular functional activity or response as possibly being mediated by a given population of 5-HT receptors. Indeed, binding profiles of a series of agents are commonly used to characterize newly identified 5-HT receptors. Serotonin itself binds at most populations of 5-HT receptors with low nanomolar affinity. No agent displays an absolute specificity for one population of 5-HT receptors vs. the others, and very few agents possess significant selectivity for a particular population of 5-HT receptors. Agents that have been useful in identifying and investigating the different populations of 5-HT receptors will be mentioned under the appropriate headings; a few general comments regarding serotonergic agents and their classification will be made here.

Subpopulation specificity, and indeed agonist vs. antagonist activity, is not so much a matter of chemical class as it is a matter of substituent type. lists some classes of agents that bind at 5-HT receptors. Within most of these different classes can be found examples of agents that bind at various populations of 5-HT receptors; that is, a particular chemical class of agents is not typically associated with only one receptor population (50). Small structural changes can result in agents with greater selectivity (i.e., in more selective or in semi-selective agents). A case in point is 5-HT. 5-HT is an indolealkylamine that binds to all populations of 5-HT receptors. The simple quaternization of 5-HT to the N,N,N-trimethyl derivative 5-HTQ enhances its affinity for 5-HT3 receptors by an order of magnitude, and 5-HTQ binds with low affinity at most other populations of 5-HT receptors. An appropriate example of substituent groups modifying agonist vs. antagonist activity is seen with the aminotetralin derivative 8-hydroxy-2-(di-n-propylamino)tetralin (8-OH DPAT). The R(+)-isomer of 8-OH DPAT is a full 5-HT1A agonist whereas its S(-)-enantiomer is a partial agonist. Furthermore, introduction of a 5-fluoro group [i.e., S(-)UH 301] results in an antagonist. Thus, small structural changes can influence affinity, selectivity, and agonist vs. antagonist character (49).

In order to simplify the chemical classification scheme shown in , certain bioisosteres are not specifically mentioned; however, in certain instances where the isosteres represent important classes of agents, they have been listed. For example, benzoic acid esters and amides, indolecarboxylic acid esters and amides, and other miscellaneous heteroaryl carboxylic acid esters and amides, bind at 5-HT3 (and certain other populations of 5-HT) receptors. Their reverse esters and amides (i.e., esters and amides where the ester oxygen or amide nitrogen has been moved from one side of the carbonyl group to the other) also bind. Thus, it would not be unlikely for structurally related compounds with a nitrogen atom on both sides of the carbonyl group (i.e., ureas) or an oxygen on one side and a nitrogen on the other (i.e., carbamates) to bind also. Indeed, there are numerous examples of such agents (50). Thus, although ureas and carbamates may be considered bioisosteric with their corresponding esters and amides, they too are listed separately in for purposes of clarity.

Indolealkylamines such as tryptamines and ergolines are notoriously nonselective agents, but, as mentioned above, small structural changes can result in enhanced selectivity. Notwithstanding their general nonselectivity, ergolines typically bind at all populations of 5-HT receptors (except for 5-HT3 and 5-HT4) and constitute an important class of agents, in that various examples bind with very high affinity at many populations of 5-HT receptors. Consequently, given their large size and shape and their stereochemically defined nature, they can reveal useful information about 5-HT receptors (27). In addition, tritiated and radioiodinated forms of various ergot derivatives (e.g., [3H]mesulergine, 2-[125I]iodo-LSD) are frequently used to label the different populations of 5-HT binding sites. Arylpiperazines lacking a piperazine N4 substituent (i.e., simple arylpiperazines) are usually nonselective; however, depending upon the presence and nature of an N4 substituent, arylpiperazines (i.e., N4-substituted or long chain arylpiperazines) bind with high affinity and/or greater selectivity at, in particular, 5-HT1A and 5-HT2A receptors. The nature of this N4 substituent may also influence intrinsic activity. The alkyl- and arylpiperidines include agents such as the 5-HT1A/5-HT2A antagonist spiperone (which also binds at 5-HT7 receptors), and the 5-HT2A/5-HT2C antagonist ketanserin (which, incidentally, binds with reduced affinity at 5-HT2B sites). Most of these agents were developed primarily as 5-HT2A antagonists, but few show any selectivity for 5-HT2A vs. 5-HT2C receptors. Aryloxyalkylamines, such as the b-adrenergic antagonists propranolol and pindolol, bind at 5-HT1A and 5-HT1B receptors; with the appropriate structural modifications, aryloxyalkylamines can be made selective for 5-HT1A receptors. Arylbiguanides constitute a rather poorly investigated class of agents; to date, the few arylbiguanides that have been studied appear to be selective for 5-HT3 receptors. Recent studies suggest that the intact biguanide structure is unnecessary for 5-HT3 binding and agonist activity (36); that is, arylguanidines retain 5-HT3 receptor affinity and agonist activity. The keto compounds constitute one of the largest chemical categories of serotonergic agents. Most of these compounds were developed as 5-HT3 ligands (primarily antagonists); however, a number of these agents have been shown in recent years to be 5-HT1P or 5-HT4 agonists or antagonists. Small structural modifications have resulted in newer agents that appear selective for 5-HT4 vs. 5-HT3 receptors.

With the recent discovery of some of the newer populations of 5-HT receptors, the selectivity of the few "selective" agents comes into question. It still remains to be seen how examples of the different classes of agents bind at some of the newly identified sites.

Several general overviews of 5-HT receptors have been published that cover various aspects, and to varying degrees, some of the material to be discussed (64,66,86,127,128,158). Descriptions of some 5-HT ligands, as well as the medicinal chemistry and structure-activity relationships (SAR) of serotonergic agents, are also available (50,52,64).

5-HT1A receptors were originally defined as those 5-HT1 sites labeled in rat brain homogenates by [3H]5-HT that displayed high affinity for spiperone; 5-HT1 sites with low affinity for spiperone were termed 5-HT1B. Nearly coincidental with the discovery of these sites was the development of a novel serotonergic agent: 8-hydroxy-2-(di-n-propylamino)tetralin (8-OH DPAT). Although it required several years, 8-OH DPAT was subsequently identified as a 5-HT1A-selective (relative to 5-HT1B and 5-HT2A) agent and [3H]8-OH DPAT was introduced as a radioligand for labeling 5-HT1A sites. To date, this compound remains one of the most selective serotonergic agents available (but see 5-HT7 Receptors). Although a number of other radioligands have been explored over the years, [3H]8-OH DPAT still remains a popular radioligand for labeling 5-HT1A sites.

The regional distribution of 5-HT1A receptors in the brains of various animal species has been examined. The highest densities are in the hippocampus, septum, amygdala, and cortical limbic areas. 5-HT1A receptors located in the raphe nuclei correspond to somatodendritic autoreceptors. 5-HT1A receptors are negatively coupled to an adenylate cyclase second messenger system. There is also evidence that some 5-HT1A receptors are positively coupled to adenylate cyclase; this may be accounted for either by the existence of different types of 5-HT1A receptors or the coupling of 5-HT1A receptors to different G-proteins (158). The possibility also exists that positively-coupled 5-HT1A receptors identified in early studies might actually be the newer 5-HT4 receptors or perhaps, due to the significant affinity of 8-OH DPAT, 5-HT7 sites (e.g., 145).

Rat, mouse, and human 5-HT1A receptors were among the first 5-HT receptors to be cloned. Like other G protein-coupled receptors, and like all other cloned 5-HT receptors with the exception of 5-HT3 receptors, the 5-HT1A receptors consist of seven transmembrane (TM) helices connected by intra- and extracellular loops (see for a schematic representation of a generalized G-protein receptor structure, and for a specific example of a 5-HT2A receptor model showing the seven helices and interconnecting loops). 5-HT1A receptors differ significantly from most other 5-HT receptors and exhibit a substantial similarity to adrenergic receptors; this may explain why a number of adrenergic agents, including propranolol, pindolol, oxymetazoline, and WB-4101, bind at 5-HT1A receptors with high affinity. For additional discussion of cloned 5-HT1A receptors, see references 17,109, 127, 135, and 158.

The membrane-spanning portions of rat and human 5-HT1A receptors exhibit a high (99%) degree of homology (62). The results of various receptor cloning experiments have led to several generalizations regarding receptor similarity (62). Any two receptors whose amino acid sequences are about 70–80% identical in their membrane-spanning segments may have highly similar to nearly indistinguishable pharmacological profiles and/or second messenger systems. Such closely related receptors (i.e., an intermediate-homology group) can be considered members of the same subfamily. In addition, there is a low-homology group (35–55% TM homology) that consists of distantly related receptor subtypes from the same neurotransmitter family, and there is also a high-homology group (95–99% TM homology) that consists of species homologs from the same gene in different species (62). Furthermore, species homologs of the same gene reveal high sequence conservation in regions outside the transmembrane domains, whereas intraspecies receptor subtypes are usually quite divergent (62). Because ligand binding to G-protein coupled receptors likely occurs in the membrane-spanning regions, similarities in these regions may account for the non-selectivity of various ligands. For example, human 5-HT1A receptors exhibit only 50–55% TM homology with 5-HT1D receptors, and yet one of the early problems in identifying 5-HT1D-selective agents was their high affinity for 5-HT1A receptors. On the other hand, even a high sequence homology between two receptors does not preclude the identification of "selective agents". For example, spiperone binds at 5-HT2A sites vs. 5-HT2C sites with about 1000-fold selectivity even though the two receptors share 78% sequence homology.

The prototypic 5-HT1A receptor agonist ligand is the aminotetralin derivative 8-OH DPAT; numerous derivatives of 8-OH DPAT have now been reported (e.g., 82). One of the problems encountered with certain aminotetralins is their reduced bioavailability, which has led to efforts to develop novel aminotetralins with greater oral availability (e.g., 81,141). Long-chain arylpiperazines probably represent the largest class of 5-HT1A ligands, and such agents bind with modest to very high affinity at 5-HT1A receptors (50). Some of the more widely used arylpiperazine (partial) agonists include buspirone (the first arylpiperazine approved for clinical use), gepirone and ipsapirone. SAR have been described for this class of agents and there appears to be considerable structural latitude for 5-HT1A binding (50); this probably accounts for the large number of compounds that have been developed. Arylpiperazine fluorescent probes (12) and candidates for single photon emission computed tomography (SPECT) imaging (78) have also been developed. The general chemical structure for these arylpiperazines is Ar-PIPERAZINE-(CH2)n-Terminus where Ar is an aryl or heteroaryl group attached to N1 of the piperazine moiety, (CH2)n is a spacer attached to N4, and the Terminus is usually an amide or imide function. With respect to n two to four methylene groups appear optimal. This chain length (n) can influence affinity and selectivity. When n = 4, optimal affinity is associated with a terminus that contains a heteroarylamide; however, when n = 2 alkylamides seem optimal (49,50,104). Although long-chain arylpiperazines typically possess an amide or imide moiety at or near the chain terminus, neither the amide or imide moiety is required for binding; that is, the terminus may be a phenyl or some other aromatic group (49). Even the presence of an intact piperazine moiety is unessential for binding or buspirone-like pharmacological activity (92). The Ar group is typically a substituted phenyl or heteroaromatic moiety (e.g., 2-pyrimidinyl); it is rather curious, however, that the presence of an oxygen atom at the phenyl 2-position (e.g., 2-methoxyphenyl) usually, but not always (91), imparts antagonist character. 2-Methoxyphenylpiperazine derivatives with structural similarity to buspirone, such as BMY 7378 and NAN-190, were the first novel agents shown to possess 5-HT1A antagonist activity (49). Eltoprazine is another arylpiperazine that has been used as a 5-HT1A ligand; structurally related agents such as flesinoxan (120) have been developed, and eltoprazine has served as a lead for the development of a number of other agents (147). Application of some of the above mentioned SAR to eltoprazine has resulted in S15535, an agent that acts as a postsynaptic 5-HT1A antagonist and a presynaptic agonist (107). Hundreds of arylpiperazine derivatives have been synthesized in order to explore their agonist vs. antagonist character and their therapeutic potential (50). Some arylpiperazines are nonselective and may variously bind at other populations of 5-HT, dopamine, and adrenergic receptors. Certain aminotetralins (e.g., UH-301) and arylpiperazines (e.g., WAY 100135) represent new classes of 5-HT1A antagonists (see below). The alkylpiperidine spiperone is a 5-HT1A antagonist but, as already mentioned, lacks selectivity.

There are three types of agents to be considered: (a) 5-HT1A agonists and partial agonists, (b) 5-HT1A antagonists, and (c) mixed 5-HT1A + X ligands where X is typically 5-HT2, dopamine and/or some other class of receptors. Further consideration must be given to whether 5-HT1A ligands act primarily at pre- or postsynaptic loci. 5-HT1A ligands with agonist activity seem to possess antianxiety, antidepressant, antiaggressive, and perhaps anticraving, anticataleptic, antiemetic, and neuroprotective properties (reviewed: 33). 5-HT1A receptors may be involved in impulsivity and alcoholism (33,39) and in the different phases of sleep (97). The main therapeutic potential of 5-HT1A receptors has been in the treatment of anxiety and depression. The application of animal models to investigate the role of 5-HT1A receptors in these disorders has been reviewed (39). Buspirone is clinically available as an anxiolytic agent; indeed, a number of structurally-related agents hold the promise of being novel anxiolytics (111). 5-HT1A agents might also be useful in the treatment of depression (138), and there may be a relationship between serotonin metabolism, depression, and violent behavior. The antianxiety actions of 5-HT1A (partial) agonists may involve primarily presynaptic somatodendritic 5-HT1A receptors (leading to reduced release of 5-HT in terminal areas), whereas the antidepressant action of 5-HT1A agents may primarily involve postsynaptic 5-HT1A receptors (33). Certain 5-HT1A agents display antiaggressive behavior, and measurement of the density of 5-HT1A receptors in frontal cortex of suicide victims reveals that nonviolent suicide victims had a significantly higher Bmax, compared with controls and violent suicides (87). The presence of alcohol is also associated with a decreased density of 5-HT1A receptors in certain brain regions (34). In the first clinical trial of its kind, gepirone was found to produce marked improvement in several depressed patients (115). Buspirone was effective in the treatment of mixed anxious-depressive patients (144). 5-HT1 (5-HT1A ?) receptors may be involved in obsessive-compulsive disorders. 5-HT1A receptors may also be involved in sexual behavior, appetite control, thermoregulation, and cardiovascular function (128,138).

To date, most investigations have employed agents that are either 5-HT1A agonists or partial agonists. A new direction in 5-HT1A research targets the development of 5-HT1A antagonists (41). Agents such as spiperone, (-)propranolol and (-)pindolol were the first to see application as 5-HT1A antagonists; however, these agents bind at least as well at other populations of neurotransmitter receptors as they do at 5-HT1A receptors. The next generation of 5-HT1A antagonists included agents such as BMY 7378 and NAN-190; however, it has now been demonstrated that these agents possess postsynaptic antagonist character, but presynaptic agonist action (41,49). There is also evidence that NAN-190 may produce some of its effects via a non-5-HT1A receptor mechanism (122) and may also be a 5-HT7 antagonist (84). Although such agents are still in use, a third generation of agents—"silent antagonists" (antagonists that lack any agonist actions)—has been developed. WAY 100135, a structural relative of BMY 7378 and NAN-190, and S(-)UH-301, a derivative of 8-OH DPAT, have been identified as being both pre- and postsynaptic 5-HT1A antagonists (41). Despite the dopamine agonist actions of S(-)UH-301, it is an effective 5-HT1A antagonist. WAY 100635 may be a more suitable antagonist than WAY 100135, because the latter, but not the former, exhibits 5-HT1A partial agonist activity (9!popup(ch39ref9)). WAY 100635 may also be an inverse agonist at 5-HT1A receptors (99!popup(ch39ref99)). A recent and rather interesting observation is that silent 5-HT1A antagonists, such as WAY 100135 and S(-)UH-301, are not intrinsically inactive and can indirectly produce non-5-HT1A 5-HT-mediated actions (30,96); presumably, blockade of 5-HT1A autoreceptors increases the postsynaptic concentration of 5-HT that leads to activation of other 5-HT receptor populations. Human evaluation of silent and selective 5-HT1A antagonists should prove interesting and could open new vistas in 5-HT1A research. For example, pretreatment of patients with agents possessing 5-HT1A antagonist character may accelerate the onset of effects of selective serotonin reuptake inhibitors (SSRIs) and enhance their clinical efficacy (8).

Various 5-HT1A ligands, particularly the arylpiperazines, can also bind at other populations of receptors (e.g., 5-HT2, dopamine, adrenergic, sigma). Attempts have now been made to turn this seeming disadvantage to an advantage. For example, the 5-HT1A/5-HT2 antagonist S 21357 is being examined as a panic-modulating agent (57), and the 5-HT1A agonist/5-HT2 antagonist FG5865 suppresses alcohol intake by rats (83). Mixed 5-HT1A agonists/5-HT2 and D2 antagonists are being evaluated as antipsychotic agents (100,101) and RWJ-37796 (mazapertine), an agent with high D2/D3/5-HT1A/a1A-adrenergic affinity may represent a novel class of antipsychotic agents with low potential for extrapyramidal effects (117). Another attempt to avoid extrapyramidal stimulation typically associated with antipsychotic agents is the exploration of mixed 5-HT1A/s receptor ligands (16).

5-HT1B receptors were one of the first 5-HT1-like receptors to be described (53,158). It was later shown that the distribution and second messenger coupling of 5-HT1B receptors in rodent brain was similar to that of 5-HT1D receptors in mammalian brain, leading to speculation that 5-HT1B and 5-HT1D receptors might constitute species variants of the same receptor (158). 5-HT1B receptors were initially identified in rodent brain using radioligand binding techniques and were defined as sites labeled by [3H]5-HT with low affinity for spiperone. 5-HT1B receptors are located presynaptically, where they control the release of 5-HT, and postsynaptically, where the highest density of 5-HT1B receptors in rat and mouse brain is found in the substantia nigra, globus pallidus, and dorsal subiculum (158. The synthesis of 5-HT1B receptors at the level of serotonergic terminals has been demonstrated (35). 5-HT1B receptors are negatively coupled to adenylate cyclase. Rat and mouse 5-HT1B receptor genes have been cloned. The cloned mouse 5-HT1B receptor (which has been referred to as 5-HT1Bb) exhibits 100% identity to the rat 5-HT1B receptor in the transmembrane region and differs overall by a total of only five amino acids. This will be further discussed below.

Early studies on ligand selectivity for 5-HT1B receptors can be rather confusing. Much of the early work was done at a time when only two populations of 5-HT1 sites were recognized: 5-HT1A and 5-HT1B. Because it was possible to mask binding at 5-HT1A sites, residual binding was presumed to be attributable to 5-HT1B receptors. Most agents originally found to be 5-HT1B-selective are now known to be fairly non-selective. For example, the simple arylpiperazine 1-(3-trifluoromethylphenyl)piperazine (TFMPP) was long used as a 5-HT1B-selective agonist, but it is now known that TFMPP binds at multiple populations of 5-HT receptors with less than 10-fold selectivity. Further adding to their nonselective nature, TFMPP and RU 24969 (another agent long used as a "selective" 5-HT1B agonist) have been shown to enhance release of 5-HT. To date, no 5-HT1B-selective ligands have been identified. Two of the more selective agents are the RU 24969 analog CP-93,129 (and related derivatives) and serotonin O-carboxymethylglycyltyrosinamide. A radioiodinated version of the latter compound has been used in autoradiographic and radioligand binding studies. Both of these agents also bind to 5-HT1D receptors. Aryloxyalkylamines such as propranolol and pindolol bind at 5-HT1B receptors and, under the appropriate masking conditions, tritiated iodocyanopindolol has been used to label 5-HT1B sites. The aryloxyalkylamine isomoltane binds at 5-HT1B receptors and appears to be an antagonist; in contrast propranolol and pindolol are partial agonists. For further discussion of these and related agents, see references (53) and (158); for comparisons of binding profiles and intrinsic activities, see reference (131).

Rodent 5-HT1B receptors play a role in thermoregulation, respiration, appetite control, sexual behavior, aggression, and anxiety (53). Past studies, however, utilized agents that are now recognized as lacking selectivity for 5-HT1B receptors. In addition, the possible existence of multiple populations of 5-HT1B receptors, and the relationship between 5-HT1B and 5-HT1D receptors, have raised new questions. The functional significance of 5-HT1B receptors begs re-examination, but such studies must await the development of agents with greater selectivity. Nonetheless, recent studies support a role for 5-HT1B receptors in the regulation of sleep, sensorimotor inhibition, and to some extent, locomotor activity (94,136). Another method for obtaining information about 5-HT1B receptors is by use of 5-HT1B receptor knock-out mice (85,155). Such mutant mice failed to display any obvious developmental or behavioral deficit but supported earlier suggestions that 5-HT1B receptors might be involved in locomotor activity and aggressive behavior (113). At one time, it was thought that 5-HT1B-type actions in rodents might be extrapolated to 5-HT1D-like actions in humans; however, it was later suggested that due to differences in receptor structure, rat 5-HT1B receptors may be poor models for the development of human drugs (62). Nevertheless, there are some behavioral similarities between the effect of 5-HT1B systems in rodents and 5-HT1D systems in other mammals (136,155) that deserve further study.

5-HT1D receptors were first identified in bovine caudate by radioligand binding techniques. They are widely distributed throughout the central nervous system (53,158), are G protein-linked, and are coupled to inhibition of adenylate cyclase. A 5-HT1D-like receptor, termed 5-HT1R, has been identified in rabbit brain; little work has been done with this population (which may actually represent several populations).

Due to the manner in which binding studies were conducted (i.e., by masking other 5-HT1-like receptors known at that time), there was early speculation about the possible existence of additional 5-HT1-like receptors, or of 5-HT1D receptor heterogeneity. Indeed, further investigation eventually led to the discovery of 5-HT1E receptors. The controversy surrounding the possible existence of 5-HT1D receptors in rat brain further confounded 5-HT1D research. This situation was subsequently remedied by molecular biology. A canine 5-HT1D receptor (RDC4) and a human 5-HT1D receptor were cloned and found to display about 90% amino acid sequence homology in their transmembrane domains (62). Hartig et al. (62) suggested that the genes encoding these receptors are species homologs and termed the cloned human 5-HT1D receptor 5-HT1Da. A human gene encoding a second 5-HT1D receptor has also been cloned, and this second receptor was termed 5-HT1Db. The human 5-HT1Da and 5-HT1Db receptors display about 77% sequence homology; however, their pharmacological properties are nearly indistinguishable (62). The gene encoding 5-HT1Da has been localized to chromosome 1, whereas that encoding 5-HT1Db is located on chromosome 6 (31,69). Cloning and characterization of rat and mouse 5-HT1B receptors revealed a high degree of similarity with human 5-HT1Db receptors. A mouse 5-HT1B receptor (5-HT1Bb), identical to rat 5-HT1B receptors in the transmembrane region, exhibits >90% homology with human 5-HT1Db receptors, only 59% homology with human 5-HT1Da receptors, but is distinct from a second mouse receptor (5-HT1Ba?) that possesses 89% homology with human 5-HT1Da receptors. It appears, then, that human brain expresses two closely related intraspecies 5-HT1D receptors (5-HT1Da and 5-HT1Db), whereas in rat and mouse the equivalent genes encode receptors whose pharmacological properties are associated with two separate pharmacological sites (5- HT1D and 5-HT1B) [62]. That is, human 5-HT1Da receptors appear to have species equivalents in dog (RDC4), rat (5-HT1D) and mouse (5-HT1Ba), as do human 5-HT1Db receptors in rat (5-HT1B) and mouse (5-HT1Bb). Functionally, the difference between rat 5-HT1B receptors and human 5-HT1Db receptors seems to be due to the presence of a threonine residue at position 355 (i.e., T355) in transmembrane helix 7 (TM7) of the latter and the presence of an asparagine residue at the corresponding position in 5-HT1B receptors; site-directed mutagenesis studies by several groups of investigators have demonstrated that conversion of T355 to an asparagine (i.e., a T355N mutant) can account for the binding differences of certain ligands (e.g., aryloxyalkylamines such as propranolol; see references 3 and 54 for further discussion). Combined ligand SAR, site-directed mutagenesis, and molecular modeling studies have led to the conclusion that, although most typical serotonergic agonists bind in the central cavity formed by TM3 (i.e., TM helix 3), TM4, TM5 and TM6 (Site 1), propranolol most likely occupies the region defined by TM1, TM2, TM3, and TM7 (). The higher affinity of propranolol for the T355N mutant 5-HT1Db receptor, relative to the wild-type, was specifically attributed to the formation of two hydrogen bonds between the receptor asparagine and the ether and hydroxy oxygen atoms of propranolol ( and ) [54].

Many 5-HT1D receptors studies were conducted prior to the discovery of 5-HT1Da and 5-HT1Db receptors. Thus, binding data for many agents necessarily reflect "overall 5-HT1D" character, and, in many instances, these agents have not been reevaluated at the two individual subpopulations. To date, there are few 5-HT1D-selective ligands. The one agent commonly referred to as a 5-HT1D agonist is sumatriptan (5-HT1D Ki = 20-30 nM). However, sumatriptan binds at 5-HT1D receptors with only 2- to 20- fold selectivity over 5-HT1A and 5-HT1B receptors, and also binds at 5-HT1Eb and 5-HT1F sites. Its affinity for 5-HT1Da sites (Ki = 5.8 nM) and 5-HT1Db (Ki = 7.7 nM) sites is nearly identical (153). Structure-activity relationships for the binding of various agents at 5-HT1D receptors have been reported (53). Many indolealkylamines bind with high affinity but with little selectivity. Interestingly, however, the aryloxyalkylamine (-)pindolol, which binds at 5-HT1A and 5-HT1B receptors, displays little affinity for 5-HT1D receptors (130). Yohimbine and rauwolscine bind with 50- and 250- fold selectivity at 5-HT1D vs. 5-HT1B receptors (130).

Relatively few agents have been examined at 5-HT1Da vs. 5-HT1Db receptors. Although attempts are being made to develop agents selective for 5-HT1Da vs. 5-HT1Db receptors and vice versa, little success has been achieved. For a diverse series of 19 agents, 5-HT1Da binding was highly correlated (r = 0.96) with 5-HT1Db binding but interestingly nearly all agents seemed to bind with slightly higher affinity for the 5-HT1Da receptor (153). A more significant effort has been reported on developing agents selective for 5-HT1D vs. other populations of 5-HT receptors; in particular, because sumatriptan displays some affinity for 5-HT1A receptors, a number of investigations have targeted development of 5-HT1D vs. 5-HT1A selectivity. Rabbit 5-HT1Da and 5-HT1Db receptors have been recently cloned; interestingly, sumatriptan binds at these receptors with lower affinity than it displays for human receptors, even though there is 91–92% amino acid sequence identity between the receptors from the two species; detailed investigations of the amino acid sequence differences may provide information on how sumatriptan binds at the molecular level (63).

Approximately a dozen new agents have been identified as displaying high affinity and reasonable selectivity for 5-HT1Da/5-HT1Db receptors (61). These include zolmitriptan (BW311C90), IS 159, MK-462, naratriptan, BMS-18004, and alniditan; LAS 31416 binds at 5-HT1D receptors. All of these agents are currently undergoing clinical trials. Of these, all are tryptamine derivatives or sumatriptan-related structures, except for the benzopyran alniditan. Other agents currently being investigated include L-694,247, NOT (ALX-1323), CP 122,288, and MDL-74,721 (an aminotetralin). L-694,247 is a highly selective 5-HT1D agonist but possesses limited oral bioavailability. NOT displays about 300-fold selectivity for 5-HT1Db receptors vs. 5-HT1A receptors and, like zolmitriptan, displays low affinity for 5-HT1E sites. CP 122,288 was found to block neurogenic dural inflammation with a potency unrelated to its affinity for 5-HT1Da or 5-HT1Db receptors. With respect to potency at blocking neurogenic dural extravasation, both of the latter compounds were up to several thousand times more potent than other 5-HT1D agonists; it has been speculated that other populations of (5-HT ?) receptors may be involved in producing this effect. IS 159 and alniditan bind with low affinity at 5-HT1F receptors whereas the affinities of sumatriptan and zolmitriptan are significant (61).

Recently, two 5-HT1D receptor antagonists have been developed: GR127935 and GR55562 (28). GR127935 in particular displays selectivity and high affinity (ca. 10 nM) for 5-HT1Da and 5-HT1Db receptors (137). Both agents antagonize many of the effects of sumatriptan (32,137), but it seems that GR127935 may possess partial agonist effects in cloned 5-HT1D receptors (61). Nevertheless, these two structurally related agents should prove quite useful in the further delineation of 5-HT1D receptor pharmacology.

The clinical significance of 5-HT1D receptors remains largely unknown. There has been speculation that these receptors might be involved in anxiety, depression, and other neuropsychiatric disorders, but this remains, for the most part, to be substantiated. With the availability of the 5-HT1D antagonists, it has been shown for example that GR127935 blocks the effect of antidepressants in the mouse tail suspension test (103). Further, the localization of 5-HT1D receptors in human brain is thought to be consistent with potential involvement in Huntington's disease (106).

Sumatriptan is clinically effective in the treatment of migraine, and logical extrapolation implies a role for 5-HT1D receptors in this disorder. However, there is considerable controversy regarding the nature of the actual 5-HT receptors involved in migraine. Sumatriptan binds nearly equally well at 5-HT1Da and 5-HT1Db receptors (153), and it also binds at 5-HT1F sites (2). This only added further fuel to the controversy. It has been variously suggested that 5-HT1Da (19), and 5-HT1Db (61) receptors should be targeted for the development of novel antimigraine agents; this may be because 5-HT1Da receptors seem to be primarily involved in neurogenic inflammation, due to their preponderance in neuronal tissue, whereas 5-HT1Db receptors may be more involved in vasoconstriction.

Using [3H]5-HT as radioligand, initial reports of 5-HT1E receptors in human cortical homogenates were based on binding studies where the masking of 5-HT1A, 5-HT1B, and 5-HT2C receptors led to biphasic competition curves for certain ligands (79). One component of these curves was associated with 5-HT1D binding, whereas the other was attributed to a new population of sites: 5-HT1E receptors. The low affinity of 5-CT and ergotamine for 5-HT1E receptors allowed their differentiation from 5-HT1D receptors. High-affinity [3H]5-HT binding was sensitive to guanine nucleotides, but functional coupling to a second messenger system was not reported. Preliminary studies also indicated the presence of 5-HT1E receptors in bovine and rat brain (79). There was relatively little interest in this population of receptors until the cloning of human 5-HT1E receptors was independently reported by three laboratories (58,89,157). The new sequence was identical to that reported earlier in 1992 (80) for a novel 5-HT receptor which, at the time, was not identified as a 5-HT1E receptor. Consistent with the results of the initial radioligand binding study (79), the cloned 5-HT1E receptors are rather unique in that they display low affinity for most serotonergic agents. Even simple O-methylation of 5-HT reduces affinity by about 100- to 300-fold (58,157). Few agents bind at 5-HT1E receptors with Ki values of <100 nM, e.g., lysergol, ergonovine, and methylergonovine (157). Functional studies, in cells stably expressing 5-HT1E receptors, indicate that the receptor is negatively coupled to adenylate cyclase. However, cloned human 5-HT1E receptors may couple to adenylate cyclase via two distinct pathways. In general, the type of second messenger pathway activated by receptors depends upon the cellular environment in which they are expressed and upon the density of receptors (e.g., 1). It has been shown, using 5-HT1E receptors transfected into BS-C-1 cells, that 5-HT produces a Gi-mediated inhibition of forskolin-stimulated cAMP accumulation at low concentrations, whereas it also elicits a significant, although with lower efficiency, potentiation of cAMP accumulation at higher concentrations due primarily to coupling to Gs (1). Methiothepin, which binds at 5-HT1E receptors only with modest affinity (Ki ca. 200 nM), is a weak competitive antagonist (89,157).

On the basis of its sequence homology (64%) to 5-HT1Da and 5-HT1Db receptors, 5-HT1E receptors may be viewed as a distantly related member of the 5-HT1D family (89,157). Support for this concept comes from site-directed mutagenesis. Human 5-HT1E receptors possess a threonine residue at a position (T330) that corresponds to the T355 of 5-HT1Db receptors; 5-HT1E receptors, like 5-HT1D receptors, display low affinity for aryloxyalkylamines such as propranolol (i.e., Ki >10,000 nM). T330N mutant 5-HT1E receptors, where the threonine has been replaced by asparagine, display up to 500-fold higher affinity for aryloxyalkylamines (3). A mouse 5-HT1E receptor has also been cloned; originally referred to as a "5-HT6" receptor it has now been re-named 5-HT1Eb due to its significant transmembrane sequence homology (62%) with human 5-HT1E receptors (6). Accordingly, it has been suggested that human 5-HT1E receptors be termed 5-HT1Ea (88). Mouse 5-HT1Eb receptors also display similarity to human 5-HT1Da (56%) and mouse 5-HT1Bb (54%) receptors (88). In addition, it appears that the mouse 5-HT1Eb receptor is a species homolog of human 5-HT1F receptors (see next section on 5-HT1F Receptors).

The newest 5-HT1 receptor to be cloned is the human 5-HT1F receptor. The 5-HT1F receptor exhibits intermediate transmembrane homology with several other 5-HT1 receptors: 5-HT1E (70%), 5-HT1Da (63%), 5-HT1Db (60%), 5-HT1A (53%) (2). Despite similarities to 5-HT1E receptors, 5-HT1F receptors bind 5-methoxytryptamine and certain ergot derivatives with high affinity (Ki <100 nM). The 5-HT1D agonist sumatriptan also binds at 5-HT1F sites; in contrast, 5-CT (Ki » 700 nM) binds only with modest affinity.

The cloned human 5-HT1F receptor couples to inhibition of adenylate cyclase. Agonist effects of 5-HT were completely, and apparently competitively, antagonized by the nonselective 5-HT antagonist methiothepin (2). Detection of 5-HT1F receptors in the uterus and mesentery suggest a possible role in vascular contraction. Although distribution in the brain appears limited, there are distributional similarities with 5-HT1Db receptors. As discussed with 5-HT1D and 5-HT1E receptors, the aryloxyalkylamines propranolol and pindolol bind with low affinity (Ki >10,000 nM) [3]. As with these other receptor populations, 5-HT1F receptors lack an asparagine in TM helix 7 and, instead, possess an alanine moiety (A333); mutant A333N 5-HT1F receptors display 300- to >1,000-fold enhanced affinity for the aryloxyalkylamines than do wild-type 5-HT1F receptors (3). There is a high degree of homology between human 5-HT1F receptors and mouse 5-HT1Eb receptors. In fact, their transmembrane regions differ only by two amino acid residues, one in TM helix 4 and one in TM helix 5. Thus, it is likely that these two receptor types are interspecies homologs.

The clinical significance of 5-HT1F receptors is unknown at this time. The binding of sumatriptan at this receptor population suggests that 5-HT1F receptors may be involved in migraine (2). Recent studies show that 5-HT1D receptors are the dominant species in human cerebral blood vessels, but they further show that 5-HT1F receptors are expressed both in neural and vascular tissue (19).

5-HT1P sites are labeled by [3H]5-HT and display a pharmacology distinct from other 5-HT receptors. 5-HT1P receptors are found in the gut but, because they have not been identified in the central nervous system, they will not be discussed here and are mentioned only for completeness (66).

Early studies suggested that 50% of sites labeled by [3H]5-HT in rat spinal cord were either 5-HT1A or 5-HT1B receptors. Subsequent investigations afforded similar results but additionally identified the remaining 5-HT receptors as a novel population that lacked the characteristics of other 5-HT sites; these were termed 5-HT1S sites (156). 5-HT1S sites display high affinity for 5-HT (Ki = 6.3 nM) and tryptamine (Ki = 6.9 nM), modest affinity (Ki = 200-300 nM) for methiothepin, cyproheptadine and metergoline, and micromolar affinity for 8-OH DPAT, TFMPP, RU 24969, propranolol, quipazine, methysergide, mianserin, ketanserin and tropisetron (ICS 205-930). Saturation studies performed in the presence of guanine nucleotides showed that GTP and Gpp(NH)p significantly reduced the number of 5-HT1S receptors labeled by [3H]5-HT, with a corresponding increase in apparent Kd values. In contrast, ATP had no effect on kinetic parameters. Although 5-HT1S receptors appear the predominant 5-HT1 receptor population in spinal cord, no significant density of 5-HT1S receptors was found in brainstem or frontal cortex. 5-HT1S receptors may be involved with modulation of pain input to the spinal cord (156).

Although 5-HT2 receptors were one of the first populations to be identified, 5-HT2 receptor research is currently in a state of transition. Numerous 5-HT2 ligands, agonists and antagonists, had been developed and 5-HT2 pharmacology was being extensively studied — then came the discovery of 5-HT2C receptors and the renaming of the original 5-HT2 receptors as 5-HT2A receptors. This prompted a shift in research focus to identify agents that would discriminate between these two subpopulations of receptors and differentiate between 5-HT2A- vs 5-HT2C-mediated pharmacological effects. In the midst of this work came the very recent discovery of a third member of the family: 5-HT2B receptors. This created another shift in research priority—the development of agents that would now discriminate between all three subpopulations of the 5-HT2 family. Some progress has been made in this direction, and this will be further discussed at the end of this section. Given these new developments, the pharmacology of the 5-HT2 system has yet to be unraveled. It is with these caveats that the three subpopulations are described.

5-HT2A, originally referred to as 5-HT2 and less commonly as 5-HT2a, receptors were among the first 5-HT receptors to be identified and, consequently, have been extensively reviewed (64,158). 5-HT2A receptors are widely distributed at varying densities throughout the brain; the highest density is in the neocortex. 5-HT2A receptors are directly coupled to a phosphoinositol second messenger system. In certain brain regions, 5-HT stimulates phospholipase A2 via a 5-HT2 mechanism.

Early on, many actions—both central and peripheral—of serotonergic agents were attributed to a 5-HT2A, as opposed to a 5-HT1, mechanism. This was due, in large part, to the availability of what was then considered a very selective 5-HT2A (vs. 5-HT1) antagonist: ketanserin. [3H]Ketanserin is still a widely used radioligand for binding studies. With the subsequent discovery of 5-HT2C receptors and the finding that many 5-HT2A ligands bind nearly equally well at both types of receptors, came the realization that some of the roles attributed to 5-HT2A receptors may in fact be mediated by 5-HT2C receptors. For example, ketanserin, the most widely used 5-HT2A antagonist, has been reported to bind with as little as 2-fold to as much as 140-fold selectivity for 5-HT2A vs. 5-HT2C receptors. Most findings, however, are closer to the lower end of this range. Agents such as DOI and DOB act as 5-HT2 agonists or partial agonists but display <10-fold selectivity for 5-HT2A vs. 5-HT2C receptors. Another confounding factor in 5-HT2 research was the initial finding that 5-HT, and other agents with 5-HT2A agonist activity, displayed low affinity for [3H]ketanserin-labeled 5-HT2A sites. Evidence was subsequently provided suggesting that 5-HT2A receptors exist in a high-affinity state and a low-affinity state; under normal conditions, this equilibrium is heavily in favor of the low-affinity state, and the tritiated antagonist ketanserin binds at both states with comparable affinity. In contrast, radiolabeled agonists such as [3H]DOB and [125I]DOI label the high-affinity state (64). Although this concept is not without controversy (158), 5-HT and the more selective 5-HT2A agonists bind with 50- to 100-fold higher affinity at agonist-labeled sites than at [3H]ketanserin-labeled sites.

5-HT2A receptors have been cloned from various species including human (125), and exhibit a high degree (>90%) of homology. In addition, there is significant (78%) homology in the transmembrane portions of 5-HT2A receptors and cloned 5-HT2C receptors. This may, to some extent, explain the observed similarities in the binding of various ligands at the two receptor populations.

The structure-activity relationships of 5-HT2A ligands have been reviewed (50,52). Most indolealkylamines, which typically bind with higher affinity at [3H]DOB- or [125I]DOI-labeled sites than at [3H]ketanserin-labeled sites, are nonselective 5-HT2A ligands. Various phenylalkylamines, such as DOB and DOI, act as 5-HT2A agonists or partial agonists, are significantly more selective than the indolealkylamines due to their low affinity for non-5-HT2 sites, but do not differentiate between 5-HT2 subpopulations.

One of the largest and most selective classes of 5-HT2A ligands are the N-alkylpiperidines. The best known example is ketanserin. Another widely used 5-HT2A antagonist is ritanserin, but ritanserin also binds at 5-HT2B, 5-HT6, and 5-HT7 sites. Although numerous ketanserin-related derivatives have been prepared, their structure-activity relationships have still not been well documented. Unlike ketanserin, which binds at 5-HT2C sites, spiperone displays about 500- to 3000-fold selectivity for 5-HT2A vs. 5-HT2C sites (see ). Spiperone has been employed as a 5-HT2A antagonist; however, spiperone is also a dopamine antagonist, a 5-HT1A antagonist, and a 5-HT7 antagonist. Various structurally related agents, such as the piperazine derivative irindalone, also act as 5-HT2A antagonists. Many 5-HT2A antagonists, although fairly selective for 5-HT2A and 5-HT2C receptors vs. most other populations of 5-HT receptors, bind with modest to high affinity at other (particularly dopaminergic, histaminergic and/or adrenergic) neurotransmitter receptors. Various tricyclic agents (e.g., tricyclic neuroleptics and tricyclic antidepressants) also bind at 5-HT2A receptors. In general, although ketanserin and spiperone are among the more widely used antagonists, and DOB and DOI are useful agonists, no truly selective 5-HT2A agent has yet been found.

Many of the clinical actions of 5-HT2A receptors may actually involve 5-HT2C receptors or a combination of 5-HT2A and 5-HT2C receptors. For the most part, the specific role of 5-HT2B receptors is unknown. The potential therapeutic roles of 5-HT2A ligands, and the possible involvement of 5-HT2A receptors, in modulating normal physiological functions and various pathological and psychopathological conditions have been extensively reviewed (e.g., 128,138,158). 5-HT2A receptors play a role in appetite control, thermoregulation and sleep. They are also involved, along with various other 5-HT receptor populations, in cardiovascular function and muscle contraction. They have also received considerable attention from a neuropsychiatric standpoint. Various antipsychotic agents and antidepressants bind with relatively high affinity at 5-HT2A receptors. Although there is no direct correlation between their receptor affinities and clinically effective doses, evidence is strong that these disorders involve, at least to some extent, 5-HT2A (or perhaps 5-HT2C) receptors. For example, chronic administration of 5-HT2A antagonists results in a paradoxical down-regulation of 5-HT2A receptors; such a down-regulation would be of benefit in the treatment of depression. Several 5-HT2A antagonist are currently in clinical trials as potential antipsychotic agents. Additionally, many 5-HT2A antagonists bind at dopamine receptors. Although this may cloud the role of 5-HT2A antagonism vs. dopamine antagonism as being more important for antipsychotic activity, it has been suggested that certain types of schizophrenia may actually be more responsive to the combined effect. Certain atypical antipsychotic agents bind both at dopamine and 5-HT2 receptors and one atypicality theory suggests that the 5-HT2 component of binding may be related to the decrease in extrapyramidal side effects associated with these types of agents. That is, those agents that display a D2/5-HT2 ratio of >1 seem to produce fewer extrapyramidal effects than those with ratios of <1. The atypical antipsychotic agent clozapine, for example, binds at D2 receptors and 5-HT2 receptor with Ki values of 100–500 nM and 10–50 nM, respectively, for a ratio of 10–20. From preclinical studies, there are indications that 5-HT2A antagonists possess anxiolytic properties. One 5-HT2A antagonist, ritanserin, produced an antianxiety effect in humans. The role of 5-HT receptors in anxiety has been reviewed (111). 5-HT2A receptors may be involved in the actions of the classical hallucinogens (48). Although indolealkylamine (e.g., 5-methoxy-N,N-dimethyltryptamine; 5-OMe DMT) and ergot-related (e.g., LSD) classical hallucinogens are fairly nonselective agents that bind at multiple populations of 5-HT receptors, the phenylalkylamine hallucinogens (e.g., DOB, DOI) are much more 5-HT2-selective. Furthermore, the human hallucinogenic potencies of classical hallucinogens and their 5-HT2A receptor affinities are significantly correlated (48). Interestingly, phenylalkylamine hallucinogens also bind at 5-HT2B and 5-HT2C receptors, and here, too, there is a significant correlation between human potency and receptor affinity for seventeen different agents (98). Recent studies suggest that 5-HT2A receptors may play a more prominent role than 5-HT2C receptors in the behavioral actions of hallucinogens (40,67,132). Nevertheless, potential roles for 5-HT2C receptor involvement require additional investigation. However, to date, there have been no attempts to block the effect of classical hallucinogens in humans with 5-HT2A antagonists.

The rat stomach fundus preparation has been used as a functional assay for serotonergic action for nearly 40 years. Long-standing questions concerning the pharmacological similarity of fundus receptors to the 5-HT2 family of receptors were answered once they were cloned (18,149,150). 5-HT2B receptors (originally termed 5-HT2F receptors, and not to be confused with 5-HT2B terminology occasionally used to describe what were once called 5-HT1C receptors or the low-affinity state of the original 5-HT2 receptors) exhibit about 70% homology to 5-HT2A and 5-HT2C receptors, and, like 5-HT2A receptors, appear to couple functionally to phosphoinositol hydrolysis. The question has been raised as to whether rat, mouse, and human 5-HT2B receptors represent homologous subtypes or species variants (26). Nevertheless, rat and human 5-HT2B receptors display >90% TM sequence homology, and most agents that bind at rat 5-HT2B receptors also bind with similar affinity at human 5-HT2B receptors. There are some exceptions, however (150). Using [3H]5-HT as radioligand, 5-HT binds with high (Ki <10 nM) affinity. The standard 5-HT2A antagonist ketanserin binds with relatively lower affinity (see ). The 5-HT2A/2C agonist DOI also binds with high affinity (Ki » 20 nM); in fact, a number of DOI-related hallucinogens have been shown to bind at 5-HT2B receptors (98). Agonists generally showed higher affinity for rat 5-HT2B receptors when binding assays were conducted at 0 °C than at 37 °C, whereas antagonists showed no difference in affinity; thus, binding at the two temperatures was found to be predictive for agonist vs. antagonist activity (149). For further discussion, see: 5-HT2-subpopulation selectivity below.

5-HT2C receptors (once referred to as 5-HT1C or, less commonly, as 5-HT2b receptors) were originally identified using autoradiographic and radioligand binding techniques (64,158). Initially characterized in porcine choroid plexus, 5-HT2C sites have now been found in low densities in various brain regions of different animal species. Human 5-HT2C receptors have been cloned (125) and display a high homology with 5-HT2A receptors. Like 5-HT2A receptors, 5-HT2C receptors are coupled to phosphoinositol hydrolysis. There is some evidence that 5-HT2C receptors may also be linked to stimulation of cGMP production. Many agents initially thought to be selective for 5-HT2A receptors were subsequently found to bind with high affinity at 5-HT2C receptors; in fact, the 5-HT2C affinities of various agents typically parallel their 5-HT2A affinities and, at this time, there are no 5-HT2C-selective agents (see ). Consequently, many pharmacological functions once attributed to 5-HT2A receptors, on the basis of their being produced by DOB or DOI and/or their being antagonized by ketanserin and related agents, may actually involve a 5-HT2C (or 5-HT2B) mechanism. For example the hyperthermic activity of a series of phenylisopropylamines is significantly correlated with their 5-HT2A affinity; later studies have shown a similar correlation between hyperthermic activity and 5-HT2C receptor affinity (48). Numerous atypical antipsychotic agents bind at 5-HT2A and 5-HT2C receptors; however, there is no correlation between their atypical nature and binding (121). 5-HT2C receptors may play a greater role than 5-HT2A receptors in migraine. On the basis of a significant correlation between migraine prophylactic activity and binding affinity, 5-HT2C receptors may be involved in the initiation of migraine attacks; however, the available evidence did not allow for a mechanistic distinction between involvement of 5-HT2C relative to 5-HT2B receptors (70). Recent attempts have been made to sort out which behaviors might be 5-HT2A-mediated, relative to those that are 5-HT2C-mediated (76). But this has been a difficult task, due to the paucity of selective agents. Spiperone, a 5-HT1A and dopamine antagonist, displays >500-fold selectivity for 5-HT2A vs. 5-HT2C sites; amperozide and pimozide also bind with comparable selectivity (121). Newer agents that might be useful in defining the potential involvement of 5-HT2A vs. 5-HT2C receptors as being important in a given functional effect are shown in . The discriminative stimulus effects of the 5-HT2 agonists DOM and DOI probably involve 5-HT2A rather than 5-HT2C receptors, because of the ability of 5-HT2A-selective antagonists to potently block their effects (67!,132). Likewise, the head-twitch response in rats is mediated by 5-HT2A, not 5-HT2C, receptors; interestingly, however, the response also requires a functionally intact D1 and D2 system and is subject to a modulatory inhibitory influence by postsynaptic 5-HT1A receptors (e.g., 133). Clearly, however, additional agents with greater selectivity are required. In the absence of such agents, evidence of a role for 5-HT2C receptors in eating disorders and epilepsy has been obtained using mice lacking these receptors. Mice generated by introducing a nonsense mutation into exon 5 of the cognate gene to produce a stop codon within the fifth putative transmembrane segment of the 5-HT2C receptor, thereby eliminating production of the carboxy-terminal half of the protein, showed an increase in body mass and had spontaneous seizures that had features in common with some forms of human epilepsy (142). The results with the mutant mice were consistent with earlier reports that the nonselective 5-HT2C agonist mCPP acts as an appetite suppressant. mCPP may also produce a discriminative stimulus effect in rats through what is primarily a postsynaptic 5-HT2C mechanism (24), although this has been disputed (21). Chronologically, 5-HT2C receptors were identified prior to the newer 5-HT2B receptors; consequently, there have been very few investigations to reexamine the binding of 5-HT2A/2C ligands at 5-HT2B sites for purpose of comparison (see next section).

To date, there are no agents that display absolute specificity for one subpopulation of 5-HT2 receptors over the others. Certain agents seem quite selective for 5-HT2A vs. the other two populations of sites, and some show 5-HT2B or 5-HT2B/2C selectivity; no 5-HT2C-selective agents have yet been identified. shows several semiselective agents with different binding profiles for the three subpopulations, but with preference for one or two subpopulations over the other(s). Spiperone, MDL 11,939, and AMI-193 are the most 5-HT2A-selective. SB 200646A was first reported to be a selective 5-HT2C antagonist with about 50-fold selectivity over 5-HT2A receptors; however, it was later shown to bind nearly equally well at 5-HT2C and 5-HT2B receptors (18). SB 200646A antagonizes DOI-induced headshake behavior in rodents and mCPP-induced hypophagia, hypolocomotion, and anxiety (71). Other preclinical studies also suggest that 5-HT2B/2C antagonists may possess anxiolytic activity (72). SB 204741 is structurally related to SB 200646A but appears to be more selective for 5-HT2B receptors (18,42). The newest member of this structural family, SB 206553, is a 5-HT2B/2C antagonist capable of blocking mCPP-induced hypolocomotion (43) and that shows an antianxiety profile in preclinical studies (73). SDZ SER-082 was one of the earlier 5-HT2B/2C antagonists with low affinity for 5-HT2A receptors (102), but it may be a partial agonist (43). SR 46349B is a 5-HT2A/2C antagonist (rat 5-HT2A IC50 = 5.8 nM; porcine 5-HT2C IC50 = 120 nM) that is inactive as an antagonist in the rat fundus preparation (118) and which, unlike most 5-HT2 antagonists, up-regulates brain 5-HT2 receptors (119). Ziprasidone (CP-88059) is another example of a mixed 5-HT2A/2C antagonist, but its affinity for 5-HT2B receptors has not been reported. The a-methyltryptamine derivative BW 723C86 represents a novel 5-HT2 agonist; this agent binds at human 5-HT2A and 5-HT2C receptors with Ki values of about 80 and 500 nM, respectively. It is reportedly an agonist in the rat fundus preparation while being only a weak 5-HT2A agonist (14).

5-HT3 receptors constituted one of the first three families of 5-HT receptors to be studied. Unlike most of the other 5-HT receptors, where functional assays were slow to be identified (and, in some cases, have yet to be identified), early 5-HT3 pharmacology involved only the use of functional assays. Due to the pharmacological similarity between 5-HT3 receptors and 5-HT-M receptors, it might be said that 5-HT3 research actually began in the 1950s; however, the first radioligand binding studies were not reported until thirty years later. 5-HT3 receptors are unique among the families of 5-HT receptors, in that they are nonselective Na+/K+ ion channel receptors. They are found in the periphery and also in the central nervous system, particularly in the area postrema, entorhinal cortex, frontal cortex, and hippocampus (158). Differences in agonist potencies and efficacies, and antagonist potencies, in various functional assays led to early speculation about 5-HT3 receptor heterogeneity or at least interspecies variation among 5-HT3 receptors. Furthermore, the distribution of 5-HT3 binding sites in human brain was not identical to that in rodent brain, and the results of radioligand binding studies using postmortem human brain revealed differences from other species. Support for interspecies differences has come from molecular biological studies. 5-HT3 receptor cDNA was initially isolated from a mouse neuroblastoma cell line; later, a splice variant was isolated. Mouse brain cDNA encodes for a shorter isoform, and rat 5-HT3 cDNA has also been isolated. Recently, human 5-HT3 receptors (hippocampus, amygdala) have been cloned (for further discussion see refs. 15 and 90). As might be expected, these human 5-HT3 receptors are similar in structure to nicotinic acetylcholine receptors, another member of the ion channel superfamily of receptors. Interestingly, differences in binding profiles have been found using these cloned human receptors, compared with rodent receptors (see below).

As expected, many indolealkylamines bind at 5-HT3 receptors in a nonselective manner. Ergolines do not bind or bind only with very low affinity. 5-HT is a nonselective 5-HT3 agonist that binds only with modest affinity (Ki = ca. 500 nM). 2-Methyl 5-HT is a somewhat more selective agent that binds with slightly lower affinity than 5-HT; although it may be only a partial agonist, it has found widespread application in 5-HT3 research due to its greater selectivity. The N,N,N-trimethyl quaternary amine analog of 5-HT (5-HTQ) binds with about ten times the affinity of 5-HT, is much more selective than 5-HT or 2-methyl 5-HT but, due to its quaternary nature, will likely not penetrate the blood-brain barrier. Phenylbiguanide (PBG) is another example of a low affinity (Ki ca. 1,000 nM) 5-HT3 agonist. Its 3-chloro derivative mCPBG (metachlorophenylbiguanide) binds in the low nanomolar range and retains agonist character. The 2,3,5-trichloro derivative binds with significantly higher affinity (Ki = 0.44 nM) (95) but has not been extensively investigated. Newer phenylbiguanide-related agents, such as (3-chlorophenyl)guanidine (mCPG; Ki = 35 nM)) and (2-naphthyl)guanidine (Ki = 25 nM), act as 5-HT3 agonists. This implies that an intact phenylbiguanide structure is unnecessary for 5-HT3 binding or agonist activity (36). Arylpiperazines were among the first serotonergic agents investigated at 5-HT3 receptors. Many arylpiperazines are nonselective 5-HT3 ligands; some behave as 5-HT3 agonists, some as partial agonists, and some as antagonists, depending on the particular substitution pattern and perhaps on the assay system being used (e.g., see 36 and 114 for discussion). The 5-HT2 agonist quipazine binds with high affinity at 5-HT3 receptors (Ki » 1 nM) and [3H]quipazine was actually used for a short while as a radioligand for labeling 5-HT3 receptors. Appropriate structural modification of arylpiperazines can result in rather selective 5-HT3 partial agonists and antagonists (7,114).

MDL 72222 was the first selective 5-HT3 antagonist. Its development resulted from the structural modification of cocaine, an agent previously shown to be a weak 5-HT-M receptor antagonist. Since then, numerous 5-HT3 antagonists have been identified. Many of these agents belong to the structural class of compounds referred to as keto compounds (). Some of the more widely used or newer agents include ondansetron, tropisetron (ICS 205-930), zacopride, granisetron, renzapride, zatosetron, dolasetron, bemesetron, and WAY-100579. Many keto compounds also bind at 5-HT1P and/or 5-HT4 receptors. The structure-activity relationships of 5-HT3 ligands have been reviewed (50,52) and those of 5-HT3 antagonists have been particularly well investigated (reviewed: 74) .

Presumably homooligomeric, cloned 5-HT3 receptor subunits function as channel receptors. Using cloned mouse 5-HT3 receptors, 5-HT and 5-HTQ act as full agonists, suggesting that the quaternary nature of 5-HTQ has little effect on efficacy. 2-methyl 5-HT and tryptamine act as partial agonists; kinetic and other studies suggest that full agonists and partial agonists may recognize distinct conformations of the receptor (146). PBG and mCPBG also act as full agonists at mouse and rat receptors (90,146) but are less potent at human 5-HT3 receptors; conversely, 2-methyl 5-HT, which behaves as a partial agonist at rodent receptors, acts as a full agonist at human 5-HT3 receptors (15,90). Further, mCPBG binds with about 200-fold lower affinity at human vs. rat 5-HT3 receptors, whereas 2-methyl 5-HT binds with about 10-fold higher affinity at rat receptors (90). With the cloning of human 5-HT3 receptors, many of the standard agents may require reexamination.

5-HT3 antagonists have proven clinically effective for the treatment of chemotherapy-induced or radiation-induced nausea and vomiting (60), whereas they are ineffective against motion sickness and apomorphine-induced emesis (56). There are also indications that they may be effective in the treatment of migraine or the pain associated with migraine. Preclinical studies suggest that 5-HT3 antagonists may enhance memory and be of benefit in the treatment of anxiety, depression, pain, and dementia (56). 5-HT3 antagonists may represent a novel class of atypical antipsychotics; however, additional clinical trials are required to substantiate these claims (56). 5-HT3 receptors can control dopamine release and may also be involved acetylcholine release and control of the GABAergic system (158). This intimate relationship could explain some of the pharmacological properties of 5-HT3 ligands. Interestingly, phenylbiguanide induces carrier-mediated release of [3H]dopamine independent of a 5-HT3 mechanism (129). Furthermore, dopamine itself acts as a 5-HT3 partial agonist (146). Finally, there is evidence that 5-HT3 antagonists may suppress the behavioral consequences of withdrawing chronic treatment with drugs of abuse, including alcohol, nicotine, cocaine, and amphetamine (56). One of the most attractive features of 5-HT3 antagonists is their general lack of undesirable side effects characteristic of many psychotherapeutic agents. Very little is known about the possible therapeutic application of 5-HT3 agonists; it seems that some partial agonists possess an anxiolytic profile (114).

A novel population of 5-HT receptors, originally identified in primary cell cultures of mouse embryo colliculi, were later called 5-HT4 receptors (158). 5-HT4 receptors have a broad tissue distribution and are positively coupled to adenylate cyclase. In the brain, 5-HT4 receptors are localized on neurons and may mediate slow excitatory responses to 5-HT. In collicular and hippocampal neurons, 5-HT4 receptors stimulate adenylate cyclase, and the mechanism by which these receptors inhibit K+ channels in collicular neurons involves cAMP production and consequent activation of cAMP-dependent protein kinase A. Peripherally, 5-HT4 receptors facilitate acetylcholine release in guinea pig ileum and may play a role in peristalsis.

Until very recently, 5-HT4 receptors were conspicuous for being one of the very few populations which had yet to be cloned. Rat 5-HT4 receptors have now been cloned and display low transmembrane sequence homology (<50%) with other 5-HT receptors (46). In fact, two isoforms have been isolated: a long form (5-HT4L) with 406 amino acids, and a short form (5-HT4S) with 387 amino acids. These isoforms, likely splice variants, differ only in their C-terminus ends. They possess identical transmembrane regions (i.e., the sequence is identical between positions 1 and 359) [46], although their tissue distribution differs somewhat. Binding profiles obtained with rat 5-HT4L and 5-HT4S receptors, using [3H]GR113808 as radioligand, were generally similar to those previously reported for guinea pig and human brains. Further, the rank order of potency was very similar for 5-HT4L, 5-HT4S and guinea pig caudate, but somewhat different from human caudate (46). In general, the potency of agonists to stimulate cAMP release was greater for the 5-HT4S receptor than for the 5-HT4L receptor; whether these receptors couple to different isoforms of Gs and/or adenylate cyclase is not yet known. It had been suggested on the basis of experiments with rat brain homogenates that 5-HT4 receptors might exist in high- and low-affinity states, as described above for 5-HT2 receptors. Support for this concept has come from binding studies with the antagonist radioligand [3H]GR113808 and the agonist radioligand [3H]5-HT using the rat 5-HT4L receptors (4).

Although 5-HT3 receptors represent ion channel receptors, whereas 5-HT4 receptors represent G-protein coupled receptors, much of the early work in 5-HT4 research was facilitated by 5-HT3 ligands; that is, a number of 5-HT3 ligands were shown to be active at 5-HT4 receptors. Even more interesting is that a number of 5-HT3 antagonists, or what were considered at the time to be 5-HT3-selective antagonists, actually showed 5-HT4 agonist activity. Even today, there is considerable structural similarity among various 5-HT3 and 5-HT4 ligands. In addition to lack of selectivity for 5-HT4 vs. 5-HT3 receptors, many early 5-HT4 ligands suffered from several other disadvantages such as affinity for s1 and/or s2 receptors, difficulty in crossing the blood brain barrier, and/or hydrolytic instability. For example, tropisetron, the first agent to see wide use as a weak 5-HT4 antagonist, is a potent 5-HT3 antagonist, the 5-HT4 antagonists SDZ 205-557 and DAU6285 exhibit similar affinities for 5-HT3 and 5-HT4 receptors, RS23597 binds at s1 receptors, BIMU1 is a 5-HT4 agonist but 5-HT3 antagonist, the 5-HT4 agonists BIMU8 and RS66331 display equal or higher affinity at 5-HT3 receptors than for 5-HT4 receptors, the 5-HT4 (partial) agonists zacopride, cisapride, renzapride are 5-HT3 antagonists, and esters SDZ 205-557, RS23597, SB 204070, and GR113808 have limited activity in vivo due to hydrolysis (38). RS56532, essentially a conformationally-constrained analog of zacopride, is a rather interesting agent; whereas R-RS56532 is more potent than its S-isomer as a 5-HT3 antagonist, S-RS56532 is more potent than R-RS56532 as a 5-HT4 agonist (37).

The following agents are characterized as 5-HT4 agonists: RS67333, RS67506, RS66331, RS56532, BIMU1, BIMU8, ML 10302, SB 205149, SC-53116, and its racemate SC-49518. Agents with antagonist activity include: SB 204070, GR125487, GR113808, RS39604, RS67532, RS23597, SDZ 205-557, and DAU6285 (38). Except for 5-HT3 receptors, 5-HT4 receptors appear to be the only other class of 5-HT receptors that bind quaternary amines; SB 205149, the n-butyl quaternary analog of renzapride, acts as a 5-HT4 agonist (38). A new class of 5-HT4 agonists and partial agonists (the carbazimidamides) reportedly contains some of the most potent agonists described to date (23). The most potent and selective 5-HT4 antagonists fall into two classes: benzoate esters (such as SB 204070) and indole esters (such as GR113808); SB 207266A was the first orally active 5-HT4 antagonist, primarily due to incorporation of an amide linkage rather than an ester (44,151).

Using cloned rat 5-HT4 receptors, tropisetron behaved as a silent antagonist whereas the partial agonists such as cisapride, BRL-24924 and zacopride acted as full agonists; receptor reserve was suggested as a possible explanation (46). 5-HT4 agonists also displayed 4- to 20-fold higher affinity for agonist labeled vs. antagonist-labeled 5-HT4L receptors; however, under the same conditions, tropisetron behaved in a manner consistent with that of a weak partial agonist, i.e., it displayed a 3-fold higher affinity for agonist-labeled sites (4).

The previous lack of selective agents has hampered investigations of clinical and even preclinical significance; however, with the availability of newer agents, some new information has become available. 5-HT4 agents are being examined both for their peripheral effects (e.g., irritable bowel syndrome, gastroesophageal reflux) and for their central effects. With respect to the latter, it has been suggested that 5-HT4 agonists may restore deficits in cognitive function and that 5-HT4 antagonists may be useful as anxiolytics or in the treatment of dopamine-related disorders. 5-HT4 receptors may be involved in memory and learning, and they are markedly decreased in patients with Alzheimer's disease (38). However, use of highly potent and selective 5-HT4 agonists might result in cardiovascular side effects (38). A high density of 5-HT4 receptors in the nucleus accumbens has led some to speculate that these receptors may be involved in the reward system and may influence self-administration behavior. For example, GR113808 reduces alcohol intake in rats, and it ethanol-induced reinforcing properties in rats may involve, at least in part, 5-HT4 receptors (105).

A functional mouse 5-HT receptor expressed primarily in the CNS was identified in 1992. The 5-HT5 amino acid sequence is not closely related to other 5-HT receptors, but the pharmacological properties of this receptor reportedly resemble somewhat those of 5-HT1D receptors. A closely related mouse receptor has also been cloned: 5-HT5B, leading to the re-naming of the original 5-HT5 receptor as 5-HT5A (88). The two 5-HT5 receptors exhibit 77% amino acid sequence homology but less than 50% homology with other cloned 5-HT receptors (43). The 5-HT5A gene is on mouse chromosome 5, whereas the 5-HT5B gene is on chromosome 1. Rat and human 5-HT5 receptors have also been cloned. To some extent, the 5-HT5 receptors appear to resemble 5-HT1 receptors (e.g., high affinity for 5-HT and 5-CT), however their low homology with other 5-HT1 receptors, together with the failure to demonstrate G-protein coupling, suggested that they represent a distinct family of receptors. There is a report of G-protein coupling for the rat 5-HT5B receptor, but activation of the receptor does not appear to involve cAMP accumulation or phosphoinositide turnover (154). Some have suggested that 5-HT5 receptors may utilize a novel (perhaps an ion channel) second messenger system. Recently, however, it was shown that, although 5-HT5A receptors are weakly detected on neurons in the cortex, their primary site of expression is non-neuronal (25). Rat 5-HT5A receptors are expressed in vitro and in vivo by astrocytes and are negatively coupled to adenylate cyclase (25). A human 5-HT5A, but not 5-HT5B, receptor has been identified; the human 5-HT5A receptor gene, like the rat and mouse counterparts, contain two coding exons separated by a single large intron (116). Hydropathy analysis indicates seven transmembrane-spanning helical units. The ligand binding characteristics of the human 5-HT5A receptor are generally similar to those of rat and mouse 5-HT5A receptors. Interestingly, however, methiothepin binds with about 60-fold higher affinity (Ki ca. 1 nM) at human than at mouse 5-HT5A receptors (116). The 5-HT1A ligand 8-OH DPAT reportedly binds with higher affinity at rat 5-HT5B receptors (Ki = 46 nM) (154) than at mouse 5-HT5B receptors (Ki ca. 400 nM) (88); 8-OH DPAT does not bind with significant affinity at mouse or human 5-HT5A receptors (Ki >1,000 nM) [116].

5-HT5A and 5-HT5B receptors are both labeled with [125I]I-LSD; rat 5-HT5B and human 5-HT5A receptors have been labeled with [3H]5-CT. 5-HT binds with modest affinity (Ki = 100–250 nM), whereas 5-CT binds with about 10-fold higher affinity at both receptors. Ergotamine and methiothepin bind with high affinity at human 5-HT5A receptors, whereas agents such as spiperone, sumatriptan, yohimbine, ketanserin, propranolol, zacopride and clozapine bind with much lower affinities (Ki >1,000 nM) [116]. The pharmacological function of 5-HT5 receptors is currently unknown; it has been speculated that, on the basis of their localization, they may be involved in motor control, feeding, anxiety, depression, learning, memory consolidation, adaptive behavior, and brain development (88,116,154). 5-HT5A receptors may also be involved in a neuronally-driven mechanism for regulating astrocyte physiology, with relevance to gliosis; disruption of 5-HT neuron-glial interactions may be involved in the development of certain CNS pathologies, including Alzheimer's disease, Down's syndrome, and some drug-induced developmental deficits (25). Suggestions continue to be made that 5-HT5 receptors may be involved in certain functions previously attributed to 5-HT1D receptors; the possibility exists, however, that because 5-HT5 and 5-HT1D receptors are clearly distinct from one another, earlier studies characterizing 5-HT1D pharmacology may have pooled the three receptors (25).

A cDNA encoding a novel G protein-coupled 5-HT receptor that appeared to be localized exclusively in the CNS was cloned from rat brain (93,123). This receptor, termed 5-HT6, exhibits only 36–41% transmembrane homology with 5-HT1A, 5-HT1B, 5-HT1D, 5-HT1E, 5-HT2A, and 5-HT2C receptors. Both [125I]I-LSD and [3H]5-HT label 5-HT6 sites and 5-HT displays an affinity of 151 and 56 nM, respectively. With [125I]I-LSD, certain ergots bind with high affinity (e.g., lisuride), whereas others (e.g., mesulergine) bind with only micromolar affinity. 5-Carboxamidotryptamine binds only with modest affinity. A number of typical and atypical neuroleptic agents and tricyclic antidepressants bind with Ki values of <100 nM. In HEK-293 cells stably transfected with this receptor, 5-HT produced a potent, dose-dependent increase in cAMP levels. As such, this was the first cloned 5-HT receptor shown to be coupled to activation of adenylate cyclase. 5-Methoxytryptamine and 5-CT behaved in a similar manner, lisuride acted as a partial agonist, and amoxapine, clozapine and methiothepin act as antagonists. 5-HT6 receptors may be involved in neuropsychiatric disorders (93). More recently, the human 5-HT6 receptor has been cloned; its gene structure, distribution, and pharmacology are similar to those of the rat receptor (75). Interestingly, comparison of rat and human 5-HT6 receptors led to the discovery of an apparent frame shift in the latter and led to its resequencing. Rat and human 5-HT6 receptors display 95% homology and differ in structure mostly at their carboxy-terminus end, with the human receptor being two amino acids longer than the rat homolog. Both receptors contain two introns at corresponding positions (75). Like the rat receptor, the human receptor is positively linked to adenylate cyclase. The pattern of 5-HT6 mRNA expression in human brain indicates highest levels in the caudate nucleus.

5-HT binds at human 5-HT6 receptors with moderate affinity (Ki = 65 nM), and one of the highest affinity agents is methiothepin (Ki = 0.4 nM). Agents that bind at human 5-HT6 receptors with Ki <50 nM include 5-methoxytryptamine, bromocryptine, octoclothepin, clozapine, olanzapine, chlorpromazine, loxapine, and fluphenazine (75). Agents with Ki >500 nM include 5-CT, RU-24969, sumatriptan, quipazine, ketanserin, 8-OH DPAT, haloperidol, risperidone, and mesulergine (75). A number of antipsychotic agents, typical and atypical, as well as antidepressants have been shown to bind at rat 5-HT6 receptors with low nanomolar affinity (121).

The exact clinical significance of 5-HT6 receptors is unknown at this time. The high affinity of various antipsychotics, atypical antipsychotics in particular, and antidepressants, suggest a possible connection between 5-HT6 receptors and certain psychiatric disorders (121). The different binding profiles of atypical antipsychotics may be responsible for their atypical nature (e.g., D2/5-HT2 ratio); for example, certain agents such as clozapine may be classified as atypical on the basis of their binding with higher affinity at 5-HT2A than at D2 receptors. However, those antipsychotics that produce the fewest extrapyramidal side effects in humans (e.g., clozapine, olanzapine, fluperlapine) possess high affinity for 5-HT6 receptors (75). Clozapine also selectively labels what appear to be 5-HT6 receptors (47). Risperidone, which produces extrapyramidal symptoms, binds with 1000-fold higher affinity at 5-HT2A than 5-HT6 receptors; thus, the affinity of agents for 5-HT6 receptors may contribute to the difference between typical and certain atypical antipsychotics (75). Repeated intracerebroventricular administration of antisense oligonucleotides to rats to prevent expression of 5-HT6 receptors produced a behavioral syndrome that involved an increase in cholinergic function; this led to speculation that one of the roles of 5-HT6 receptors may be to control cholinergic neurotransmission, and that 5-HT6-selective antagonists could be useful in the treatment of anxiety and memory deficits (20,140).

The 5-HT7 receptor, expressed mainly in the CNS, has been cloned from several species including rat (124,134), mouse (112), guinea pig (145) and human (13). A low level of expression of 5-HT7 receptors has been detected in the periphery (85). Alternative splicing probably accounts for two splice variants in the rat; both forms of the receptor, the genes for which contain at least two introns, are positively coupled to adenylate cyclase (85). The short form of the rat 5-HT7 receptor contains 435 amino acids whereas the long form, as well as mouse 5-HT7 receptors, contains 448 amino acids; the human 5-HT7 receptor falls in between, with 445 amino acids (85). There is <50% transmembrane sequence homology between 5-HT7 receptors and other 5-HT receptors. Gene mapping studies have led to localization of the human 5-HT7 receptor gene to chromosome 10 (45).

5-CT consistently binds at 5-HT7 receptors with subnanomolar affinity, and [3H]5-CT, as well as [3H]5-HT and [3H]LSD, have been used to label 5-HT7 binding sites. Agents with Ki values of <10 nM include 5-HT, 5-methoxytryptamine, LSD, methiothepin and mesulergine. Agents with Ki values in the 10–100 nM range include: 8-OH DPAT, spiperone, ritanserin, metergoline, mianserin, LY215840, chlorpromazine; in the 100–1,000 nM range: NAN-190, sumatriptan, haloperidol; and Ki >1,000: 2-methyl 5-HT, tropisetron, pindolol, and ketanserin. Serotonin, 5-CT, 5-methoxytryptamine, and 8-OH DPAT reportedly act as agonists, whereas methiothepin, mianserin, mesulergine, ritanserin, spiperone, NAN-190, LY215840 and clozapine act as antagonists.

Numerous antidepressants and antipsychotic agents bind at rat 5-HT7 receptors with nanomolar or subnanomolar affinity. Agents with Ki values of <10 nM include: fluphenazine, acetophenazine, chlorprothixene, zotepine, clorotepine, clozapine, fluperlapine, pimozide, tiospirone, and risperidone (121).

5-HT7 receptors might be involved in mood and learning, as well as in neuroendocrine and vegetative behaviors. The 5-HT2 ligand ritanserin, tricyclic antidepressants (e.g., amitriptyline), classical antipsychotic agents (e.g., chlorpromazine), and nonclassical antipsychotic agents (e.g., clozapine, loxapine) bind with Ki values of <100 nM (121,134). It is thought that 5-HT7 receptors may play a role in certain psychiatric disorders. A binding assay has been proposed to measure 5-HT7 receptors in rat hypothalamus, and 5-HT7 receptors were reported to be down-regulated after chronic administration of fluoxetine (139). Such results further strengthen the case for a role of 5-HT7 receptors in depression. A subsequent report, however, suggested that the assay actually reflected a heterogeneous population of receptors, and that the results should be interpreted with caution (55).

In mammals, the endogenous clock controlling circadian rhythm is located in the suprachiasmatic nuclei (SCN) of the hypothalamus. 5-HT has been shown to reset or phase-shift circadian rhythm. Although mRNAs encoding for 5-HT1B and 5-HT2C receptors are expressed in the SCN, neither of these receptor types seems to be involved in generating phase advances (84). 5-HT, 5-CT, and 8-OH DPAT are effective, suggesting that 5-HT1A receptors may be involved; likewise, the action of quipazine implicates a role for a 5-HT2 receptor subpopulation. However, the 5-HT2 agonist DOB is inactive, and the 5-HT1A antagonist pindolol does not attenuate the effect of 8-OH DPAT. 5-HT7 mRNA is expressed in the hypothalamus in neurons surrounding, and possibly within, the SCN (84). Although autoradiographic studies using [3H]5-CT have failed to find evidence of 5-HT7 receptors in the SCN (59), their existence in the hypothalamus, coupled with the realization that 5-HT, 5-CT, 8-OH DPAT, but not pindolol, bind at 5-HT7 receptors, implicate a role for this receptor population in serotonergic regulation of circadian rhythm. Thus, 5-HT7-selective agents might be effective in the treatment of jet lag or sleep disorders of a circadian nature (84).

5-HT produces both contraction and relaxation of coronary artery from various species. In canine coronary artery, contraction is mediated by 5-HT1D-like receptors, whereas relaxation seems to involve another 5-HT1-like receptor, in that it is produced by 5-CT and antagonized by methiothepin. A role for 5-HT2 receptors was eliminated because ketanserin was without antagonist activity; however, another 5-HT2 antagonist, LY53875, weakly blocked the relaxant effect of 5-HT (29). Examination of a small series of ergolines revealed that LY215840 (pA2 = 8.2) was more effective than LY53857 (pA2 = 6.3), and that antagonist potencies within the series paralleled human 5-HT7 receptor affinities. Thus, it has been proposed that relaxation of canine coronary artery may be mediated by 5-HT7 receptors (29). Another study independently reached the same conclusion. Using GR 127935 to block 5-HT1D receptors, the potencies of several agonists to produce relaxation was consistent with their affinities for 5-HT7 receptors (143). Similarly consistent was the effect of several antagonists (including methiothepin, mianserin, risperidone and spiperone) to block 5-CT-induced relaxation (143). Interestingly, in the latter study, 8-OH DPAT failed to show agonist activity and clozapine behaved as a partial agonist (143). Agents active at 5-HT7 receptors might thus be effective in the treatment of coronary heart disease.

The design of agents selective for a given population of 5-HT receptors represents a significant problem. Ideally, if the exact three-dimensional structure of a particular 5-HT receptor were known, it might be possible to design agents by targeting specific amino acid residues. Unfortunately, to date, it has not yet been possible to crystallize a membrane-bound neurotransmitter receptor to determine its x-ray crystal structure. In the absence of such information, perhaps the next best approach is the utilization of 5-HT receptor graphics models. Statistician George E. P. Box has stated: "All models are wrong, but some are useful." While this originally applied to statistical models, the statement is particularly true of molecular models of G-protein coupled receptors. Because no high-resolution experimental structures are available for G-protein coupled receptors (GPCR), model building requires some starting point to be selected and some assumptions to be made. Most GPCR models have been constructed from experimental structural data, using either bacteriorhodopsin or rhodopsin as a template for the general topology of the 7-helix aggregate (152). While the experimental structure of bacteriorhodopsin has been determined at relatively high resolution, this bacterial, light-sensitive proton pump is not G-protein coupled. A low resolution projection structure of rhodopsin provides only an indication of the potential location of the helices, but not of specific atoms or residues.

However, unlike bacteriorhodopsin, rhodopsin is a G-protein coupled photoreceptor. Unfortunately, the sequence homology between rhodopsin and the neurotransmitter GPCRs is very low. Interestingly, the shapes of the 7-helix aggregates are noticeably different in the case of bacteriorhodopsin and rhodopsin (). Despite these limitations, structural data available both for bacteriorhodopsin and rhodopsin have been used as starting templates for the construction of serotonin receptor models (152). Some of the key issues in constructing such models are identification of portions of the receptor that constitute the helical regions, the rotational orientation of each about the helix axis, and assignment of amino acid side chain conformations. The helical segments of GPCR sequences can be identified as relatively hydrophobic stretches of non-polar amino acids. More careful analysis of the distribution of hydrophobic and hydrophilic amino acid residues with in a putative helical segment usually reveals that the helices are amphiphilic. That is, hydrophilic residues and hydrophobic residues occur most frequently on opposite faces of the a-helix. It is generally assumed that the hydrophobic face of each amphiphilic a-helix should face the membrane lipid that surrounds the aggregate of helices, and that hydrophilic residues should face the water-accessible central pore, as has been demonstrated for bacteriorhodopsin. illustrates the distribution of hydrophobic and hydrophilic residues in a rhodopsin-based 5-HT2A receptor model. Similarly, highly conserved residues should lie in the central cavity or at crucial positions within the helix-helix interfaces (). The disposition of conserved proline residues may be particularly important, since they may introduce a structural kink or constitute hinged regions (). Helix amino acid side chains are usually assumed to adopt conformations shown to be most prevalent in helical segments of soluble proteins (). While it is reasonably certain which receptor sequence segments correspond to the seven helical region, virtually nothing is known experimentally about the structures of the intracellular loops. Serotonin receptor models containing the loop segments have been constructed using secondary structure prediction methods combined with the end-to-end constraints imposed by the helix termini ().

Despite the uncertainties, "wrong" GPCR models have proven "useful". As illustrated earlier in this chapter, provisional models have provided a three dimensional context for the interpretation of ligand SAR and receptor mutagenesis studies. Model building investigations can be expected to converge to the point of having utility in predicting receptor specific ligand-receptor interactions which will be useful in drug design. Indirect experimental receptor structure data, such as site-directed mutagenesis and ligand SAR, provide clues as to the nature and importance of specific receptor and ligand features in ligand-receptor interactions. However, neither type of investigation alone can conclusively indicate, for example, that a particular amino acid side chain interacts directly with of specific portion of a ligand. Since any mutation could change receptor structure in a subtle way, the observation that mutation of a specific receptor amino acid leads to a decrease in ligand affinity is not sufficient to indicate that the mutated residue resides in the ligand binding site. Experiments that can demonstrate that a specific mutation only affects the affinity of members of a series of structurally similar ligands that contain a specific ligand feature will provide better evidence that the complementary receptor and ligand features physically interact. Similarly, multiple site-directed mutations which alone adversely affect receptor function but which together restore function to a double-mutated receptor (i.e., a reciprocal mutation) provide better evidence that the two mutated receptor features interact physically. If the definition of "reciprocal mutation" is extended to include mutation of the receptor such that it now lacks affinity for the natural ligand but possesses affinity for a "mutated" ligand specifically designed to interact with the mutated receptor, this too provides information on what specific receptor (and ligand) features are important for binding. The iterative refinement of receptor models can be done by the judicious design and execution of combined modeling, ligand SAR, and receptor mutagenesis studies; investigations attempting to use such an approach include those recently published on 5-HT1A (77) and 5-HT1Db (54) receptors.

The 1987 edition of Psychopharmacology (48) described only four populations of 5-HT receptors: 5-HT1A, 5-HT1B, 5-HT1C (now 5-HT2C), and 5-HT2 (now 5-HT2A) receptors. The 1995 edition describes all the populations shown in . Thus, most of the currently known 5-HT receptor populations were identified in the intervening years. The last several years have witnessed an extraordinary number of publications (»3,000 per year) in the 5-HT area; studies have reported the cloning of several receptor populations previously known but not yet cloned (e.g., 5-HT4, human 5-HT5), the development of novel agonists and antagonists with greater subpopulation selectivity, additional molecular biological studies (e.g., site-directed mutagenesis), and additional pharmacological and clinical studies. Evidence continues to mount in support of important roles for 5-HT receptors in various neuropsychiatric disorders. Anxiety, depression, schizophrenia, migraine, and drug abuse are at the top of the list. 5-HT receptors may also play important roles in appetite control, aggression, sexual behavior, and cardiovascular disorders. As the list of 5-HT receptors grows, the number of serotonergic agents has also grown. However, more than before, there is a realization that some agents previously considered selective are not as selective as originally claimed; newer agents are required. Nevertheless, the development of novel agents is already opening new vistas. Today, we have many more selective, or semi-selective, agents than ever before. Unfortunately, pharmacologists must still rely on pharmacological profiles of a series of agents to define some receptors and must use several semiselective agents when conducting certain studies because the list of truly selective agents is very small. Knowledge of amino acid sequence data has allowed the construction of hypothetical three-dimensional graphics models of various populations of 5-HT receptors. Once appropriate models have been identified, it may be possible to rationally design novel and highly-selective serotonergic agents.

published 2000