During the COVID-19 pandemic, governments have struggled to find the right balance between restrictive measures to contain the spread of the virus, and the effects of these measures on people's psychological wellbeing. This paper investigates the relationship between limitations to mobility and mental health for British population during the COVID-19 pandemic, combining the use of high frequency mobility data from Google and longitudinal monthly data collected during the pandemic. We show that more time spent at home predicts a worsening of mental wellbeing even when we account for the prevalence of COVID-19 in the region and the general stringency of the lockdown. There is some heterogeneity in these effects, with young healthy people, living alone, with an active working life, showing particularly high levels of distress.