Despite a more recent debate about ever deeper segmentation, we argue that since industrialization, Germany has continually experienced a dual labor market. One segment contains the primary segment of better paid and more attractive jobs, while the secondary segment encompasses rather low paid, less stable and less attractive jobs. It has been argued that this dualization is the result of firms which are likely to hire full-time and long-term workforce for its core activities performed by the core workforce while relying on more flexible forms of employment for other activities. Based on an in-depth examination of the structure of the workforce since the founding of the German state, this paper seeks to explore the factors which account for the origin, evolution and the peculiarities of the country's core workforce. It will be shown that a non-negligible part of the working population has always been subjected to marginalization, but that the dividing line between the two segments has changed over time as has the character of the respective groups.