Mutations in Caenorhabditis elegans gene mab-23 cause abnormal male tail morphology and abolish male fecundity but have no obvious effect in the hermaphrodite. Here we show that mab-23 encodes a DM (Doublesex/MAB-3) domain transcription factor necessary for specific aspects of differentiation in sex-specific tissues of the male. mab-23 is required for the patterning of posterior sensory neurons in the male nervous system, sex muscle differentiation, and morphogenesis of the posterior hypodermis, spicules, and proctodeum. Failure of mab-23 mutant males to sire progeny is due primarily to defective sex muscle-mediated turning during copulatory behavior and likely compounded by impairment of sperm passage through the proctodeum. In the male nervous system, mab-23 refines ray neuron subtype distribution by restricting expression of dopaminergic neurotransmitter identity through interactions with the Hox gene egl-5 and a TGF-beta-related signaling pathway. mab-23 has distinct roles and functions independent of mab-3, indicating different aspects of C. elegans male sexual differentiation are coordinated among DM domain family members. Our results support the hypothesis that DM domain genes derive from an ancestral male sexual regulator and suggest how regulation of sexual development has evolved in distinct ways in different phyla.