arXiv cs.AI· Hai Lu, Yichen Luo, Luping Yu·· 3 天前AI 评分46
LLM 在同行评审中表现如何？来自金融与会计领域的证据
How Well Do LLMs Perform in Peer Review? Evidence from Finance and Accounting
AI 导读
GPT-4o 对 2024 年 AFA 和 NBER 金融会议论文的评审打分，比作者声誉更能预测论文能否在未来两年发表于三大金融顶刊——得分高一个标准差的论文发表概率高 44%。该信息来自模型对正确性和表述的判断，而其对原创性和重要性的判断不携带任何预测信息，强调重要性的评审指南也无帮助。这一"正确性 vs 贡献性"的分野在会计论文和更新的 GPT-5 上同样成立。
正文 · 原文
Abstract:Referees increasingly use large language models (LLMs), while finance and accounting journals disagree about whether to allow it. We ask which part of a referee's job the models can do. GPT-4o, whose training data end before the 2024 conference season, referees the papers presented at the 2024 AFA and NBER finance meetings, and we compare its verdicts with publication in the top-three finance journals over the next two years. Papers the model scores one standard deviation higher are 44\% more likely to be published, net of author reputation. The result is not driven by look-ahead bias, as the model does not recall the outcomes. The information comes from its judgements of correctness and exposition. Its judgements of originality and significance carry none, and a referee guide that stresses importance does not help. The split between correctness and contribution holds for accounting papers and for a newer model, GPT-5. LLMs can check whether research is right, but not whether it matters. The technical check can go to the model, while the judgement of contribution stays with people.
| Subjects: | General Finance (q-fin.GN); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18435 [q-fin.GN] |
| (or arXiv:2610.18435v1 [q-fin.GN] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18435 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Luping Yu [view email]
[v1]
Tue, 13 Oct 2026 14:11:35 UTC (46 KB)
来源：arXiv cs.AI · arxiv.org