TY  - RPRT
        AU  - Pataranutaporn, Pat
AU  - Powdthavee, Nattavudh
AU  - Maes, Pattie
        TI  - Can AI Solve the Peer Review Crisis? A Large-Scale Experiment on LLM's Performance and Biases in Evaluating Economics Papers
        PY  - 2025/Jan/
        PB  - Institute of Labor Economics (IZA)
        CY  - Bonn
        T2  - IZA Discussion Paper
        IS  - 17659
        UR  - https://www.iza.org/publications/dp17659
        AB  - We investigate whether artificial intelligence can address the peer review crisis in economics by analyzing 27,090 evaluations of 9,030 unique submissions using a large language model (LLM). The experiment systematically varies author characteristics (e.g., affiliation, reputation, gender) and publication quality (e.g., top-tier, mid-tier, low-tier, AI-generated papers). The results indicate that LLMs effectively distinguish paper quality but exhibit biases favoring prominent institutions, male authors, and renowned economists. Additionally, LLMs struggle to differentiate high-quality AI-generated papers from genuine top-tier submissions. While LLMs offer efficiency gains, their susceptibility to bias necessitates cautious integration and hybrid peer review models to balance equity and accuracy.
        KW  - Artificial Intelligence
KW  - peer review
KW  - large language model (LLM)
KW  - bias in academia
KW  - economics publishing
KW  - equity-efficiency trade-off
        ER  -