arXiv cs.AI· Shuhan Li, Huankang Guan, Xintong Guo, Yang Bo, Yuyang He, Dunyuan Xu, Jingyan Chen, Fang Liu, Tsz Kin Wan, Jinpeng Chen, Zhenqi He, Yuxuan Hu, Wenbo Li, Han Gao, Qijun Li, Long Chen, Hongsheng Li, Rui Liu·· 1 天前AI 评分52
AURA 2.0 发布：流式视频模型可主动发起对话
AURA 2.0: From User-Driven Responses to Model-Initiated Conversations in Streaming Video
AI 导读
AURA 2.0 是一个流式视频大语言模型，核心升级是模型主动对话（MIC），能在视频流中识别值得讨论的话题并在合适时机主动发起对话，无需用户提问。它还引入流式感知工具调用（SATC）和交互式视觉定位（IVG），后者可识别用户在视频帧上标注的对象并输出与帧对齐的边界框。
正文 · 原文
Authors:Shuhan Li, Huankang Guan, Xintong Guo, Yang Bo, Yuyang He, Dunyuan Xu, Jingyan Chen, Fang Liu, Tsz Kin Wan, Jinpeng Chen, Zhenqi He, Yuxuan Hu, Wenbo Li, Han Gao, Qijun Li, Long Chen, Hongsheng Li, Rui Liu
Abstract:Most AI assistants respond only when prompted, remaining silent even when events in the video stream offer meaningful opportunities for conversation. To move beyond purely user-driven interaction, we present AURA 2.0, a streaming video large language model that introduces three core upgrades to AURA. The key advance is Model-Initiated Conversation (MIC), which enables the model to identify topics worth discussing in the evolving visual stream and initiate conversations at appropriate moments without a user query. Stream-Aware Tool Calling (SATC) enables the model to invoke external tools using the current visual and conversational context and incorporate returned information into the ongoing interaction. Interactive Visual Grounding (IVG) supports visual grounding in both user inputs and model outputs. The model can identify objects marked by users on video frames and produce frame-aligned bounding boxes to link its responses to specific objects and regions. To develop these capabilities, we construct a dedicated streaming interaction dataset and adopt a three-stage training scheme to integrate the new behaviors with AURA's existing streaming interaction abilities. We further incorporate Long-Horizon Interaction Memory (LHIM) to maintain contextual continuity over extended video streams. Evaluations on public streaming benchmarks show that AURA 2.0 achieves leading performance among open-source streaming models, while targeted evaluations further validate its new interaction capabilities. We open-source the model weights and inference code.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.17048 [cs.CV] |
| (or arXiv:2611.17048v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17048 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shuhan Li [view email]
[v1]
Thu, 12 Nov 2026 16:31:54 UTC (16,306 KB)
来源：arXiv cs.AI · arxiv.org