Genetic contributions to dental dimensions in brown-mantled tamarins (Saguinus fuscicollis) and rhesus macaques (Macaca mulatta).

Objectives The use of dental metrics in phylogenetic reconstructions of fossil primates assumes variation in tooth size is highly heritable. Quantitative genetic studies in humans and baboons have estimated high heritabilities for dental traits, providing a preliminary view of the variability of dental trait heritability in nonhuman primate species. To expand upon this view, the heritabilities and evolvabilities of linear dental dimensions are estimated in brown-mantled tamarins (Saguinus fuscicollis) and rhesus macaques (Macaca mulatta). Materials and methods Quantitative genetic analyses were performed on linear dental dimensions collected from 302 brown-mantled tamarins and 364 rhesus macaques. Heritabilities were estimated in SOLAR using pedigrees from each population, and evolvabilities were calculated manually. Results Tamarin heritability estimates range from 0.19 to 0.99, and 25 of 26 tamarin estimates are significantly different from zero. Macaque heritability estimates range from 0.08 to 1.00, and 25 out of 28 estimates are significantly different from zero. Discussion Dental dimensions are highly heritable in captive brown-mantled tamarins and free-ranging rhesus macaques. The range of heritability estimates in these populations is broadly similar to those of baboons and humans. Evolvability tends to increase with heritability, although evolvability is high relative to heritability in some dimensions. Estimating evolvability helps to contextualize differences in heritability, and the observed relationship between evolvability and heritability in dental dimensions requires further investigation.