Existing climate-economy models use aggregate damage functions to model the effects of climate change. This approach assumes climate change has equal impacts on the productivity of firms that produce consumption and investment goods or services. We show the split between damage to consumption and investment productivity matters for the dynamic consequences of climate change. Drawing on the structural transformation literature, we develop a framework that incorporates heterogeneous climate damages. When investment is more vulnerable to climate, we find short-run consumption losses will be smaller than leading models with aggregate damage functions suggest, but long-run consumption losses will be larger. We quantify these effects for the climate damage from heat stress and find that accounting for heterogeneous damages increases the welfare cost of climate change by approximately 4 to 24 percent, depending on the discount factor.