arXiv cs.AI· Yuanhe Tian, Yan Song·· 10 小时前AI 评分27
LASER：用预期引导智能体实现视频物体状态读取
Looking One Step Ahead: An Anticipation-guided Agent for Object State Readout in Videos
AI 导读
研究者提出 LASER，一种预期引导的智能体，用于视频物体状态读取：它把对下一步变化的预判转化为针对性视觉问题，检查当前帧确定属性值，再用后续帧的独立观察校验状态读取与预测，并迭代修正状态记录以生成最终描述。整个过程模型参数保持冻结，可复用的文本观察技能通过追踪与参考描述的差异来修正。在 OSCaR 和 PLM RCap 上，LASER 相比强基线和现有方法取得一致提升。
正文 · 原文
Abstract:Understanding the progress of an activity requires tracking how the objects involved change over time. Object state readout provides this information by describing a target object's states and transitions across video frames. Although multimodal large language models (MLLMs) generate detailed visual descriptions, these descriptions may leave the attributes needed to distinguish successive states insufficiently specified. In this paper, we propose look-ahead state readout (LASER), an anticipation-guided agent for object state readout that uses possible next-step changes to determine which attributes to inspect in the present. Given temporally ordered frames and a target object, LASER converts anticipated changes into targeted visual questions and inspects the current frame to establish attribute values. It then checks state readouts and forecasts against independent observations of subsequent frames, refining a state record used to generate the final description. Reusable textual observation skills guide this process and are refined by tracing discrepancies with reference captions to the responsible instructions. Candidate revisions are selected on separate videos based on improved caption quality without increased contradictions with reference facts. Model parameters remain frozen throughout adaptation and inference, and the revised skills remain fixed during evaluation. Experiment results and further analysis on OSCaR and PLM RCap show consistent improvements of LASER over strong baselines and existing approaches, which demonstrates the effectiveness of our approach.
| Comments: | 13 pages, 2 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.14113 [cs.CV] |
| (or arXiv:2611.14113v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14113 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuanhe Tian [view email]
[v1]
Tue, 10 Nov 2026 14:05:39 UTC (7,511 KB)
来源：arXiv cs.AI · arxiv.org