We study the effects of preschool attendance on children's school progression and cognitive skills in Kenya and Tanzania. Our analysis uses novel data from large-scale household surveys of children's literacy and numeracy skills, which also collect retrospective information on preschool attendance. Against the backdrop of a large expansion of pre-primary education, our regressions identify the impacts from within-household differences, controlling for a variety of child-specific covariates. In both countries, children who go to preschool tend to enroll in primary school late, and thus fall behind in terms of grades completed at early ages. However, once in school, they progress through grades faster and at ages 13-16 have completed about one and a half more months of schooling than their same-aged peers who did not attend preschool. They also score around 0.10 standard deviations higher on standardized cognitive tests, showing that there are important longer-term benefits from preschool in Kenya and Tanzania.