In this paper, we study the development and underlying drivers of skill premiums in Germany between 1980 and 2008. We show that the significant increase in the medium to low skill wage premiums since the late 1980s was almost exclusively concentrated among the group of workers aged 30 or below. Using a nested CES production function framework which allows for imperfect substitutability between young and old workers, we investigate whether changes in relative labor supplies could explain these patterns. Our model predicts the observed differential evolution of skill premiums very well. The estimates imply an elasticity of substitution between young and old workers of about 8, between medium- and low-skilled workers of 4 and between high-skilled and medium/low-skilled workers of 1.6. Using a cohort level analysis based on Microcensus data, we find that long-term demographic changes in the educational attainment of the native (West-)German population – in particular of the post baby boomer cohorts born after 1965 – are responsible for the surprising decline in the relative supply of medium-skilled workers which caused wage inequality at the lower part of the distribution to increase in recent decades. We further show that the role of (low-skilled) migration is limited in explaining the long-term changes in relative labor supplies.