Social interaction plays an important role in early language development, and family is considered a major arena for socialization. However, little is known about the potential impact of one particular demographic group of parents, notably those parents who were themselves only children. This paper empirically examines the effect of being only-child parents on the language ability of their children. The results show that children whose parents are both only children have significantly lower language skills. We further examine urban and rural children respectively, and find that the lower language ability is mainly driven by rural children as they are more constrained by their family socio-economic background. Three channels have been identified to explore the relationship between only-child parents and the lower language ability of their children: intergenerational cognitive transmission; parental engagement in the school life of their children; children’s social relationship. Contrary to language ability, the math ability is not affected, suggesting that social interaction plays an important role in the development of language, but does not influence math cognitive skills.