arXiv cs.MA· Hangyu Qin, Junbin Xiao, Angela Yao·· 12 小时前AI 评分31
ReferClaw：为第一视角助手消解用户意图歧义
ReferClaw: Disambiguating User Intent for Egocentric Assistance
AI 导读
ReferClaw 是一个智能体框架，把第一视角流式问答改为交互过程：先由三个协作智能体识别候选解释、提出澄清问题，再基于确认的意图和视觉证据作答。其成本感知策略在预期信息增益与用户负担之间权衡，并限制澄清轮次与问题复杂度。在增强的第一视角 QA 基准上，答案准确率与答案加定位的联合表现均优于现有基线，并实现跨数据集零样本迁移。
正文 · 原文
Abstract:Users of egocentric AI assistants often ask short, ambiguous questions whose intended referents cannot be uniquely resolved from video and dialogue history. For example, "Where did I leave it?" may refer to several previously handled objects. Existing VideoQA systems typically commit to a single interpretation, risking plausible but intention-misaligned answers. We introduce ReferClaw, an agentic framework that formulates egocentric streaming QA as an interactive process: the assistant asks clarifying questions to resolve user intent before answering. Three cooperating agents identify candidate interpretations, request clarification, and generate answers grounded in the resolved intent and visual evidence. A bounded, cost-aware policy balances expected information gain against user effort while limiting clarification turns and question complexity. Referclaw learns from interaction through intent-specific skill updates and process-reward-guided policy optimization with agent-level credit assignment. Evaluations on augmented egocentric QA benchmarks demonstrate improved answer accuracy and joint answer-and-grounding performance over existing baselines, alongside zero-shot transfer across datasets. These findings support interactive clarification as a practical approach to resolving online user intent in egocentric streaming assistance.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2611.07683 [cs.CV] |
| (or arXiv:2611.07683v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07683 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hangyu Qin [view email]
[v1]
Thu, 5 Nov 2026 09:10:14 UTC (1,123 KB)
来源：arXiv cs.MA · arxiv.org