We consider the external validity of laboratory measures of risk attitude. Based on a large-scale experiment using a representative panel of the Dutch population, we test if these measures can explain two different types of behavior: (i) behavior in laboratory risky financial decisions, and (ii) behavior in naturally-occurring field behavior under risk (financial, health and employment decisions). We find that measures of risk attitude are related to behavior in laboratory financial decisions and the most complex measures are outperformed by simpler measures. However, measures of risk attitude are not related to risk-taking in the field, calling into question the methods currently used for the purpose of measuring actual risk preferences. We conclude that while the external validity of measures of risk attitude holds in closely related frameworks, this validity is compromised in more remote settings.