arXiv cs.CL· Chih-Hsuan Yang, Anjir Ahmed Chowdhury, Cheng-Hau Yang, Weijian Zheng, Fernando Llorente, Xiaolong Ma, Xinyang Li, Eliu A. Huerta, Feng Yan, Ian T. Foster, Rajeev Thakur·· 12 小时前AI 评分34
研究：多智能体消息中错误答案仍有轨迹价值，Gemma 上 73 条错答消息中 34 条获确认
Wrong but Useful: Trajectory Value Beyond Answer Correctness in Multi-Agent Messages
AI 导读
一项 arXiv 研究提出用“轨迹价值”衡量多智能体推理中消息对接收方的贡献，即固定整合器在有该缓存消息时的答对概率变化。在 Gemma 上，73 条被选中的错误答案消息中有 34 条通过预注册的新调用复测确认，超过 11.4–20.8 的对照校准误确认上限；按消息移除规则在留出回放块上比最强的仅看正确性规则提升 1.61 个百分点。
正文 · 原文
Authors:Chih-Hsuan Yang, Anjir Ahmed Chowdhury, Cheng-Hau Yang, Weijian Zheng, Fernando Llorente, Xiaolong Ma, Xinyang Li, Eliu A. Huerta, Feng Yan, Ian T. Foster, Rajeev Thakur
Abstract:A solution can end with a wrong answer and still help another solver. In multi-agent reasoning, evaluating only a message's proposed answer can therefore miss its contribution to the receiver. We measure this contribution as trajectory value: the change in a fixed integrator's probability of answering correctly when a cached message is present rather than removed. Controlled replay across five benchmarks and two model families separates message effects from identical-input variation. The strongest evidence is for Gemma: in a pre-registered fresh-call re-test, 34 of 73 selected wrong-answer messages pass individual confirmation, exceeding control-calibrated upper bounds on expected false confirmations of 11.4--20.8. The finding is benchmark-dependent and the conservative criterion no longer passes without SciBench. A blinded two-author audit supports the selected answer labels, while neutral replacements show that the helpful effect is not explained solely by message length and position. On held-out replay blocks of the same problems, a message-specific removal rule improves accuracy over the strongest tested correctness-only rule by 1.61 percentage points [1.01, 2.25]. Post hoc analysis localizes most of this gap to pools containing no correct proposal. Evidence for gpt-oss is weaker, and the tested predictors do not establish a practical way to identify useful errors in advance. The results support measuring answer correctness and receiver benefit separately, with value understood as conditional on the message pool, receiver and generation format.
| Comments: | 18 pages, 2 figures. Revised manuscript with fresh-call tests, human evaluation, and additional robustness analyses. Project website: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.14375 [cs.AI] |
| (or arXiv:2608.14375v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.14375 arXiv-issued DOI via DataCite |
Submission history
From: Chih-Hsuan Yang [view email]
[v1]
Fri, 14 Aug 2026 15:16:57 UTC (6,010 KB)
[v2]
Sat, 31 Oct 2026 04:29:01 UTC (142 KB)
来源：arXiv cs.CL · arxiv.org