This paper examines the impact of the COVID-19 pandemic on employment and respiratory health for remote workers (i.e. those who can work from home) and non-remote workers in the United States. Using a large, nationally-representative, high-frequency panel dataset from March through July of 2020, we show that job losses were up to three times as large for non-remote workers. This gap is larger than the differential job losses for women, African Americans, Hispanics, or workers without college degrees. Non-remote workers also experienced relatively worse respiratory health, which likely occurred because it was more difficult for non-remote workers to protect themselves. Grouping workers by pre-pandemic household income shows that job losses and, to a lesser extent, health losses were highest among non-remote workers from low-income households, exacerbating existing disparities. Finally, we show that lifting non-essential business closures did not substantially increase employment.