Over the last year the German government has introduced a comprehensive set of labor market policy reforms, the so-called Hartz reforms, which aim at a significant reduction of unemployment. To this end, (a) many of the existing instruments of active labor market policy are modified considerably, (b) a set of new instruments is introduced, and (c) the administrative framework in which these measures operate is changed substantially. In order to be able to judge the success of these measures by the end of the current legislative period in mid-2006, the government has asked academic experts to set up an evaluation concept capable of generating reliable empirical evidence by that date. The task is therefore to develop a ready-to-implement concept for the evaluation of the full set of reforms in their entirety, as well as each instrument on its own, facing substantive constraints regarding data availability and a short time horizon. This paper presents such a concept. We discuss essential guidelines for an ideal evaluation design, conceptual and practical difficulties that arise in the context of evaluating the Hartz reforms, and ways to overcome these obstacles. After detailing the three main analytical steps – analyses of effectiveness, efficiency, and implementation and process analysis – we present the concrete evaluation design, specific methods applicable to particular instruments, and a sampling scheme for collecting the required data. In addition to the fact that our concept is directly implementable, it also has the advantage of being extensible for future labor market policy evaluations.