The genetic mechanism involved in a transition from the black-colored seed hull of the ancestral wild rice (Oryza rufipogon and Oryza nivara) to the straw-white seed hull of cultivated rice (Oryza sativa) during grain ripening remains unknown. We report that the black hull of O. rufipogon was controlled by the Black hull4 (Bh4) gene, which was fine-mapped to an 8.8-kb region on rice chromosome 4 using a cross between O. rufipogon W1943 (black hull) and O. sativa indica cv Guangluai 4 (straw-white hull). Bh4 encodes an amino acid transporter. A 22-bp deletion within exon 3 of the bh4 variant disrupted the Bh4 function, leading to the straw-white hull in cultivated rice. Transgenic study indicated that Bh4 could restore the black pigment on hulls in cv Guangluai 4 and Kasalath. Bh4 sequence alignment of all taxa with the outgroup Oryza barthii showed that the wild rice maintained comparable levels of nucleotide diversity that were about 70 times higher than those in the cultivated rice. The results from the maximum likelihood Hudson-Kreitman-Aguade test suggested that the significant reduction in nucleotide diversity in rice cultivars could be caused by artificial selection. We propose that the straw-white hull was selected as an important visual phenotype of nonshattered grains during rice domestication.