Although non-farm enterprises are ubiquitous in rural Sub-Saharan Africa, little is yet known about their productivity. In this paper we contribute to filling this gap by providing estimates of labor productivity in enterprises for Ethiopia, Malawi, Nigeria, and Uganda. Using the World Bank's LSMS-ISA database, we find that rural enterprises are on average less productive than those in urban areas, and that female-owned enterprises are less productive than male-owned enterprises. By estimating Heckman selection and panel data models, we find that education and access to credit are associated with higher labor productivity, while households that experience shocks operate less productive enterprises. Furthermore we provide evidence that enterprises that operate throughout the year are more productive. We conclude that gender, education, shocks, access to finance, and location matter for labor productivity in rural Africa, and that policy decisions tackling the shortcomings could significantly contribute to a better business environment and increased labor productivity.