Using data for the 1990’s, this paper examines the role of sheepskin effects in the returns to education for Japan. Our estimations indicate that sheepskin effects explain about 50% of the total returns to schooling. We further find that sheepskin effects are only important for workers in small firms with the size of these effects being similar to comparable estimates for the US. These results could be explained by the particular recruitment system of large firms in Japan, which makes the university diploma as a screening device unimportant for large firms.