Using Norwegian administrative data, we examine how exposure to immigration over the past decades has affected natives' relative prime age labor market outcomes by social class background. Social class is established on the basis of parents' earnings rank. By exploiting variation in immigration patterns over time across commuting zones, we find that immigration from low‐income countries has reduced social mobility and thus steepened the social gradient in natives' labor market outcomes, whereas immigration from high‐income countries has leveled it. Given the large inflow of immigrants from low-income countries to Norway since the early 1990s, this can explain a considerable part of the relative decline in economic performance among natives with lower class background, and also rationalize the apparent polarization of sentiments toward immigration.