arXiv cs.CL· Yingqian Wu, Jingcong Liang, Shengbin Yue, Siyuan Wang, Zhenfei Yin, Junchi Yu, Zhongyu Wei·· 12 小时前AI 评分31
RAP：评估语言智能体预测研究热点的基准
RAP: Research Attention Prediction in Language Agents
AI 导读
研究者提出 Research Attention Prediction（RAP）基准，用 356,357 篇带日期的 arXiv 论文构建本地搜索环境，包含覆盖 54 个 AI/ML 领域的 2,160 道核心题和覆盖 269 个领域的扩展集。
正文 · 原文
Abstract:LLM agents increasingly support research planning and execution, where judgments about current and anticipated research activity inform decisions. Yet whether they can infer changes across a whole field from the partial evidence they retrieve through online scholarly search remains unclear. We introduce Research Attention Prediction (RAP), a literature search benchmark for evaluating this ability. We build a local search environment from 356,357 dated arXiv papers and construct two complementary evaluation tasks. The benchmark comprises a Core set of 2,160 questions across 54 AI/ML fields and an Extended set covering 269 fields. Our evaluation and analysis of LLM agents show that search substantially improves stronger agents' recognition of recent changes, while forecasting gains remain limited or negative. Agents update forecasts across dates and adjust their investigations to forecasting goals, but these adaptations do not reliably yield accurate predictions of future shifts. Even for recent changes, final judgments can diverge from what the retrieved papers indicate. These findings show that agents can recognise some recent changes, yet reliably anticipating future research activity remains difficult despite changes in their answers and investigations.
| Comments: | 30 pages, 5 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.10092 [cs.AI] |
| (or arXiv:2609.10092v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.10092 arXiv-issued DOI via DataCite |
Submission history
From: Yingqian Wu [view email]
[v1]
Wed, 9 Sep 2026 12:15:14 UTC (1,831 KB)
[v2]
Sun, 1 Nov 2026 14:24:05 UTC (855 KB)
来源：arXiv cs.CL · arxiv.org