Randomized experiments provide policy relevant treatment effects if there are no spillovers between participants and nonparticipants. We show that this assumption is violated for a Danish activation program for unemployed workers. Using a difference-in-difference model we show that the nonparticipants in the experiment regions find jobs slower after the introduction of the activation program (relative to workers in other regions). We then estimate an equilibrium search model. This model shows that a large scale role out of the activation program decreases welfare, while a standard partial microeconometric cost-benefit analysis would conclude the opposite.