For the purpose of understanding the underlying mechanisms behind intergenerational associations in income and education, recent studies have explored the intergenerational transmission of abilities. We use a large representative sample of Swedish men to examine both intergenerational and sibling correlations in IQ. Since siblings share both parental factors and neighbourhood influences, the sibling correlation is a broader measure of the importance of family background than the intergenerational correlation. We use IQ data from the Swedish military enlistment tests. The correlation in IQ between fathers (born 1951-1956) and sons (born 1966-1980) is estimated to 0.347. The corresponding estimate for brothers (born 1951-1968) is 0.473, suggesting that family background explains approximately 50% of a person's IQ. Estimating sibling correlations in IQ we thus find that family background has a substantially larger impact on IQ than has been indicated by previous studies examining only intergenerational correlations in IQ.