Eliciting expectation and introducing probabilistic questions into surveys have gained important interest. In this study, we focus on the reliability of students’ earnings expectations. To what extent is observed log earnings expectations affected by random measurement error (noise)? A test-retest method is applied and reliability is found to be fairly low; about 0.59 in 2015 and about 0.67 in 2016. Particularly homogeneous samples exaggerate problems of measurement error. The analysis show how these measures of reliability can be adjusted to become more suitable to other studies, where different degrees of homogeneity are present.