arXiv cs.AI· Babak Barazandeh, Connor Swanson, Chinmay Kulkarni, Nikhil Mungel·· 9 天前AI 评分32
OnTrack：基于流式结构感知最优传输的 LLM 智能体轨迹实时监控与干预
OnTrack: Real-Time Monitoring and Intervention in LLM Agent Trajectories via Streaming Structure-Aware Optimal Transport
AI 导读
OnTrack 是一种流式监控机制，通过将智能体步骤及依赖关系与已记录的成功运行轨迹对比，可在约每步 1 毫秒内告警或阻断智能体。在 SWE-bench 轨迹上，基于前 8 步，其区分失败轨迹的 AUROC 比内容相似度方法高 0.057；配合中止策略可节省约 18% 的失败运行算力，被中断的运行中 83% 确实走向失败。
正文 · 原文
Abstract:Agents are deployed in applications from trip planners and stock trading to IT incident triage. In most cases, LLM agents work autonomously with minimal rule-based safeguarding, leading to cost and safety issues from irreversible actions. Recent works resolve this either by using a safeguard agent to monitor behavior or evaluating logs post-hoc. The first adds cost and latency to every step; the second delivers its verdict after the run, when tokens are burned and damage is done. To overcome this, we propose OnTrack, a streaming monitoring mechanism that compares an agent's steps and dependencies against recorded successful runs to alert users or block the agent in about a millisecond per step. We study this problem in three regimes of decreasing access: full reference access (historical runs and tool schemas), intermediate access (only tool schemas), and no prior knowledge (only step logs as generated). Expectation of OnTrack's monitoring capabilities reduces as data access drops, ranging from plan violation detection to identifying loops, stalls, and repeated tool calls. Finally, we evaluate OnTrack using SWE-bench trajectories. Based on the first 8 steps, our method ranks failing trajectories below succeeding ones better than content similarity approaches (+0.057 AUROC). With an abort policy, we save about 18% of compute that would be burned on failing runs, where 83% of interrupted runs were actually heading to failure (5 out of 6 aborts were correct).
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.12375 [cs.AI] |
| (or arXiv:2610.12375v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12375 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Babak Barazandeh [view email]
[v1]
Thu, 8 Oct 2026 17:32:08 UTC (37 KB)
来源：arXiv cs.AI · arxiv.org