TY  - RPRT
        AU  - Lee, Jungmin
        TI  - Outlier Aversion in Evaluating Performance: Evidence from Figure Skating
        PY  - 2004/Aug/
        PB  - Institute of Labor Economics (IZA)
        CY  - Bonn
        T2  - IZA Discussion Paper
        IS  - 1257
        UR  - https://www.iza.org/index.php/publications/dp1257
        AB  - The quality of subjective performance evaluation is dependent on the incentive structures faced by evaluators, in particular on how they are monitored and themselves evaluated. Figure skating competitions provide a unique opportunity to study subjective evaluation. This paper develops and tests a simple model of what I call "outlier aversion bias" in which subjective evaluators avoid submitting outlying judgments. We find significant evidence for the existence of outlier aversion. Individual judges within a game manipulate scores to achieve a targeted level of agreement with the other judges. Furthermore, a natural
experiment shows that the dispersion of scores across judges depends upon the type of judge-assessment system and its implication for outlier aversion. Agreement may not be a
good criterion for the validity of an evaluation system, contradicting the industrial psychology and personnel management literature.
        KW  - subjective performance evaluation
KW  - outlier aversion bias
        ER  -