We study the effect of unemployment on birth outcomes by exploiting geographical variation in the unemployment rate across local areas in England, and comparing siblings born to the same mother via family fixed effects. Using rich individual data from hospital administrative records between 2003 and 2012, babies' health is found to be strongly pro-cyclical. A one-percentage point increase in the unemployment rate leads to an increase in low birth weight and preterm babies of respectively 1.3 and 1.4%, and a 0.1% decrease in foetal growth. We find heterogenous responses: unemployment has an effect on babies' health which varies from strongly adverse in the most deprived areas, to mildly favourable in the most prosperous areas. We provide evidence of three channels that can explain the overall negative effect of unemployment on new-born health: maternal stress; unhealthy behaviours - namely excessive alcohol consumption and smoking; and delays in the take-up of prenatal services. While the heterogenous effects of unemployment by area of deprivation seem to be explained by maternal behaviour. Most importantly, we also show for the first time that selection into fertility is the main driver for the previously observed, opposite counter-cyclical results, e.g., Dehejia and Lleras-Muney (2004). Our results are robust to internal migration, different geographical aggregation of the unemployment rate, the use of gender-specific unemployment rates, and potential endogeneity of the unemployment rate which we control for by using a shift-share instrumental variable approach.