arXiv cs.AI· Jingyu Tang, Xu Yang, Zhehao Zhang, Zheng Wang, Qi Cheng, Haozhe Zhao, Manling Li, Wenxuan Li, Xide Xia, Jiaxin Pei·· 2 天前AI 评分32
Realtime-Bench：前沿模型能否解决实时交互问题？
Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?
AI 导读
研究团队推出 Realtime-Bench，用于评测智能体在严格时间约束下的实时交互能力。在按实际推理延迟推进环境的 Wall Clock 设定下，最佳成功率仅 44%，最佳 Pass^3 仅 36%。研究还对比了排除推理延迟的 Paused Clock 与按生成 token 数推进模拟时间的 Token Clock，发现时间成本制约了动态环境中的表现，且提高推理投入在时间压力下效果不一。
正文 · 原文
Abstract:Real-time interaction requires agents to perceive evolving environments and act under temporal constraints, yet existing evaluations of real-time streaming models focus on conversational quality and streaming perception, leaving effective action under strict temporal constraints untested. We introduce Realtime-Bench, a suite of carefully curated dynamic tasks where delayed actions directly affect success. Under Wall Clock, where environments advance according to actual inference latency, the best success rate reaches only 44% and the best Pass^3 is only 36%, demonstrating that real-time interaction remains challenging. To separate serving latency from models' real-time streaming capability, we compare Wall Clock with Paused Clock, which removes inference delays as an idealized reference, and introduce Token Clock, which advances simulated time according to generated token counts, testing temporal awareness independently of physical serving speed. We find that temporal costs limit success in evolving environments and that higher reasoning effort has mixed effects under time pressure, reflecting a trade-off between decision quality and timely execution. These findings suggest directions for improving real-time multimodal agents, including adaptive reasoning with temporal awareness and continuous perception and action.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.22155 [cs.AI] |
| (or arXiv:2610.22155v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22155 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jingyu Tang [view email]
[v1]
Thu, 15 Oct 2026 16:20:47 UTC (5,954 KB)
来源：arXiv cs.AI · arxiv.org