Numeracy skills of adults within and across 12 different countries in 2011 are strongly associated with the accumulated public investments in education received by these adults during their schooling. This paper confirms existing evidence that the timing of educational investments is important, with early investments playing the most fundamental role. Investment in primary education is associated with higher numeracy scores for those who went on to continue their education. Higher investments in tertiary education are needed in order to fully realize the benefit of the investments in primary school. Family background is a decisive factor in relation to numeracy skills of these adults, in line with all available evidence. Adults who received higher public investment in primary education were more likely to complete secondary school and attain tertiary education. This refutes earlier studies indicating that the amount of financial resources available for education may not be that important for the development of competences.