We investigate the effect of having a child on parents' criminal behavior using rich administrative data from Brazil. Fathers' criminal activity sharply increases by up to 10% during the pregnancy period, and by up to 30% two years after birth, while mothers experience only a transitory decline in criminal activity around childbirth. The effect on fathers lasts for at least six years and can explain at least 5% of the overall male crime rate. Domestic violence within the family also increases after childbirth, reflecting both increases in actual violence and women's propensity to report. The generalized increase in fathers' crime stands in sharp contrast with previous evidence from developed countries, where childbirth is associated with significant and enduring declines in criminal behavior by both parents. Our findings can be explained by the costs of parenthood and the pervasiveness of poverty among newly formed Brazilian families. Consistent with this explanation, we provide novel evidence that access to maternity benefits largely offsets the increase in crime by fathers after childbirth.