It is well established that boys perceive themselves to be better in mathematics than girls, even when their ability is the same. We examine the drivers of this male overconfidence in self-assessed mathematics ability using a longitudinal study of twins. This allows us to control for family fixed effects, i.e. shared genetic and environmental factors, and exploit the random assignment of the sex of one's co-twin. Using measures of individual self-assessment in mathematics from childhood and adolescence, along with mathematics levels and test scores, cognitive skills, parent and teacher mathematics assessments, and characteristics of their families and siblings, we examine potential channels of the gender gap. Our results confirm that objective mathematics abilities only explain a small share of the gender gap in self-assessed mathematics abilities, and the gap is even larger within opposite-sex twin pairs. We find that having a confident male co-twin increases the confidence of boys but decreases the confidence of girls, not just in mathematics, but also in their self-assessment of other abilities. Male overconfidence might explain why men self-select into top jobs or STEM courses, making entry more difficult for women. We also find that parents are more likely to overestimate boys' and underestimate girls' mathematics abilities. Gender-biased parental assessments explain a large part of the gender gap in mathematics self-assessment, highlighting the importance of the intergenerational transmission of gender stereotypes.