Using a nationally representative 24-hour diary survey covering the first two years of the pandemic, we explore the mechanisms underlying the changes in wellbeing for men and women. We exploit the variation in the stringency of social restrictions implemented by the UK government during this period and use an event-study methodology to net out the impact of social restrictions from other pandemic effects. We find that well-being dropped by 47% (men) and 70% (women) of a standard deviation during the strictest lockdown, and this effect survives after accounting for financial conditions and changes in local infection and death rates. Our data on time allocation and individual preferences over the activities undertaken throughout the day reveal that the drop in well-being is primarily driven by a drastic reduction in time spent in leisure with non-household members or outside the home.