arXiv cs.AI· Zihao Lin, Haoyu Dong, Haobo Xu, Chi-Chih Chang, Xuying Ning, Yinglong Xia, Shuang Yang, Zhuokai Zhao, Xinying Song, Dongfang Liu, Lifu Huang, Yang Song, Qifan Wang·· 12 小时前AI 评分32
VideoHarness-SRSI：面向视频理解 harness 的结构化递归自我改进框架
VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness
AI 导读
研究者提出 VHarness-SRSI，一个以证据治理为核心的递归自我改进框架，围绕冻结的视觉语言模型演化视频理解 harness。在四个视频理解基准上，其等权平均留出分数相对 VHarness-BRSI 提升 6.2%，平均验证-测试差距缩小 37.9%。该框架通过限制提议者仅使用无内容反馈、用带监测的预热实验引导探索，并在隔离的 probe-race 分支中开发机制后再做因子组合与联合精修。
正文 · 原文
Authors:Zihao Lin, Haoyu Dong, Haobo Xu, Chi-Chih Chang, Xuying Ning, Yinglong Xia, Shuang Yang, Zhuokai Zhao, Xinying Song, Dongfang Liu, Lifu Huang, Yang Song, Qifan Wang
Abstract:LLM system capabilities depend not only on model parameters but also on the executable harnesses that organize information acquisition, tool use, reasoning, and memory. Recursive harness evolution can automate their design, but how it can reliably discover generalizable systems from limited, noisy, and repeatedly reused feedback remains unclear. We study this question in agentic video understanding, where rich multimodal evidence and coupled mechanisms make harness optimization challenging. We first introduce VHarness-BRSI, a recursive self-improvement framework that evolves video harnesses around a frozen vision-language model. Analyzing over 4,000 evolved candidates, we find it competitive with hand-engineered systems but identify three interconnected failure modes: validation overfitting, search-budget dilution, and the early-experience trap. We trace all three to a common root: the same imperfect validation evidence drives both harness revisions and the proposer's search strategy. We therefore propose VHarness-SRSI, an evidence-governed framework in which every search decision rests on evidence that is transferable, sufficient, and gathered before the search commits. It restricts the proposer to content-free feedback, informs exploration with instrumented warm-up experiments, and develops mechanisms in isolated probe-race branches before factorial composition and joint refinement. Across four video understanding benchmarks, VideoHarness-SRSI improves the equally weighted mean held-out score by 6.2% relative to VideoHarness-BRSI and reduces the mean validation-test gap by 37.9%. Reliable harness evolution thus depends on how search is organized and informed, not only on how well it optimizes validation performance.
| Comments: | 50 pages, 15 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.19424 [cs.CV] |
| (or arXiv:2610.19424v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19424 arXiv-issued DOI via DataCite |
Submission history
From: Zihao Lin [view email]
[v1]
Wed, 14 Oct 2026 02:17:07 UTC (690 KB)
[v2]
Fri, 16 Oct 2026 03:45:20 UTC (687 KB)
来源：arXiv cs.AI · arxiv.org