Using a unique dataset we study both the actual and self-perceived relationship between subjective well-being and income comparisons against a wide range of potential comparison groups, enabling us to investigate a broader range of questions than in previous studies. In questions inserted into a 2008 module of the German-Socio Economic Panel Study we ask subjects to report (a) how their income compares to various groups, such a co-workers, friends, and neighbours, and (b) how important these income comparisons are to them. We find substantial gender differences, with income comparisons being much better predictors of subjective well-being in men than in women. Generic (same-gender) comparisons are the most important, followed by within profession comparisons. Once generic and within-profession comparisons are controlled for, income relative to neighbours has a negative coefficient, implying that living in a high-income neighbourhood increases happiness. The perceived importance of income comparisons is found to be uncorrelated with its actual relationship to subjective well-being, suggesting that people are unconscious of its real impact. Subjects who judge comparisons to be important are, however, significantly less happy than subjects who see income comparisons as unimportant. Finally, the marginal effect of relative income on subjective well-being does not depend on whether a subject is below or above the reference group income.