arXiv cs.AI· Ningxin Pan, Hanyu Li, Yeqing Shen, Ning Ding, Yehui Tang, Wentao Zhang·· 2 天前AI 评分39
RealtimeBench：在实时运行的游戏环境中评测与训练 LLM 智能体
The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock
AI 导读
研究者推出 RealtimeBench，包含六类游戏共 180 个任务，在游戏持续运行、推理耗时的情况下评测 LLM 智能体。配对评测显示，GPT-6.1-sol 在游戏暂停时优于更快的 Jev，但游戏持续运行时反被 Jev 超越。
正文 · 原文
Abstract:Large language model (LLM) agents can solve complex interactive tasks, yet reasoning and tool calls take time. In continuously changing environments, an otherwise correct action may arrive too late. We introduce RealtimeBench, with 180 tasks across six game types, to evaluate agents while games continue running during inference. Paired evaluations show that GPT-6.1-sol outperforms the faster Jev when games are paused, but falls behind when they keep running. We propose Two-Timescale Offline Recursive Self-Improvement (TORSI): a slow teacher studies failures and tests corrections in paused games between attempts, then writes reusable instructions for a fast student. The student acts without teacher calls during gameplay, and validation scores determine which updates are adopted. Two rounds raise Jev's average test score from 13.51 to 24.19 on a 0--100 progress scale, improving all six game types. The instructions also improve average scores for three other models without retuning. These findings point toward real-time intelligence: enabling powerful LLM agents to remain effective in a continuously changing world by organizing deliberation, learning, and action across timescales.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20665 [cs.AI] |
| (or arXiv:2610.20665v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20665 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ningxin Pan [view email]
[v1]
Wed, 14 Oct 2026 09:35:54 UTC (1,951 KB)
来源：arXiv cs.AI · arxiv.org