arXiv cs.CL· Lorenzo Masoero·· 23 小时前AI 评分36
TriAgent:面向文档维护的多智能体系统在 43 天影子运行中分诊 499 张工单
Mind the Gap: A Deployed Multi-Agent System for Documentation Maintenance
AI 导读
研究团队部署了以分诊智能体 TriAgent 为核心的多智能体系统,用于持续提取显式与隐式信号并修复文档漂移。43 天影子运行中系统分诊 499 张工单、发现 92 处文档缺口,独立模型家族的 LLM 评审小组判定其可验证断言 81.1%(245/302)正确;在长度可比的回答对上,评审小组 70.5% 偏好 TriAgent、23.2% 偏好人类工程师。
正文 · 原文
Abstract:The "AI revolution" has enabled teams across the technology industry to deliver software features at unprecedented velocity. While code evolves faster than ever, customer-facing documentation often struggles to stay up to speed. As users increasingly rely on LLM-based agents that consume documentation as their primary interface, the impact of this documentation gap worsens: stale information propagates and is treated as authoritative. Signals for identifying where documentation has fallen behind already exist in abundance, both explicit (e.g., resolved support tickets revealing what documentation lacks) and implicit (e.g., code-documentation discrepancies detectable from automated loops). We deploy a multi-agent system, built around a triage agent we call "TriAgent," that continuously extracts these signals and repairs documentation drift. Over 43 days of shadow-mode operation, our system triaged 499 tickets and surfaced 92 gaps in the documentation. Using a panel of LLM judges from independent model families, 81.1% of TriAgent's verifiable assertions were correct (245 of the 302 assertions the human resolution addresses; majority of three judges); on length-comparable pairs the panel preferred TriAgent's response in 70.5% of pairs and the human engineer's in 23.2% (the rest had no judge majority). Importantly, agent- and human-sourced replies tended to be complementary, with the human often surfacing insights that were either buried within the existing documentation or missing from it entirely; our system turns each such insight into a documentation gap and, after human review, a fix. Repairing a gap measurably improved the agent's later answer: when the agent re-answered the same ticket after the repair, blinded judges preferred the post-repair answer over its own pre-repair answer 71.4% of the time (n=35). A blinded human audit confirms strong alignment on self-improvement.
| Comments: | Accepted at IAAI-27 (Innovative Applications of AI, co-located with AAAI 2027), Emerging Applications track. 12 pages (6 pages + references and appendix), 2 figures, 6 tables |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.23142 [cs.SE] |
| (or arXiv:2611.23142v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23142 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lorenzo Masoero [view email]
[v1]
Tue, 17 Nov 2026 17:59:13 UTC (45 KB)
来源:arXiv cs.CL · arxiv.org