We empirically assess the effect of historical slavery on the African American family structure. Our hypothesis is that female single headship among blacks is more likely to emerge in association not with slavery per se, but with slavery in sugar plantations, since the extreme demographic and social conditions prevailing in the latter have persistently affected family formation patterns. By exploiting the exogenous variation in sugar suitability, we establish the following. In 1850, sugar suitability is indeed associated with extreme demographic outcomes within the slave population. Over the period 1880-1940, higher sugar suitability determines a higher likelihood of single female headship. The effect is driven by blacks and starts fading in 1920 in connection with the Great Migration. OLS estimates are complemented with a matching estimator and a fuzzy RDD. Over a linked sample between 1880 and 1930, we identify an even stronger intergenerational legacy of sugar planting for migrants. By 1990, the effect of sugar is replaced by that of slavery and the black share, consistent with the spread of its influence through migration and intermarriage, and black incarceration emerges as a powerful mediator. By matching slaves' ethnic origins with ethnographic data we rule out any influence of African cultural traditions.