Worker productivity depends not only on hours worked, but also on how work time is actually used, and time-use evidence shows that non-work at work is non-trivial. This paper provides a data-driven characterization of shirking, and studies which observable characteristics best predict shirking behavior using American Time Use Survey data over 2003–2024. We implement a machine-learning forward selection procedure based on out-of-sample predictive performance. Our results suggest that shirking strongly depends on stochastic or unobserved factors, and that the determinants of the extensive and intensive margins are different. Moreover, the most informative predictors are predominantly job-related and time-allocation variables, whereas macro and labor-market indicators seem less relevant. This suggests that policies or managerial approaches to improve worker efficiency relying on observables face important limitations.