Does the death penalty save lives? A surge of recent interest in this question has yielded a series of papers purporting to show robust and precise estimates of a substantial deterrent effect of capital punishment. We assess the various approaches that have been used in this literature, testing the robustness of these inferences. Specifically, we start by assessing the time series evidence, comparing the history of executions and homicides in the United States and Canada, and within the United States, between executing and non-executing states. We analyze the effects of the judicial experiments provided by the Furman and Gregg decisions and assess the relationship between execution and homicide rates in state panel data since 1934. We then revisit the existing instrumental variables approaches and assess two recent state-specific execution moratoria. In each case we find that previous inferences of large deterrent effects based upon specific samples, functional forms, control variables, comparison groups, or IV strategies are extremely fragile and even small changes in specifications yield dramatically different results. The fundamental difficulty is that the death penalty – at least as it has been implemented in the United States – is applied so rarely that the number of homicides that it can plausibly have caused or deterred cannot be reliably disentangled from the large year-to-year changes in the homicide rate caused by other factors. As such, short samples and particular specifications may yield large but spurious correlations. We conclude that existing estimates appear to reflect a small and unrepresentative sample of the estimates that arise from alternative approaches. Sampling from the broader universe of plausible approaches suggests not just "reasonable doubt" about whether there is any deterrent effect of the death penalty, but profound uncertainty – even about its sign.