arXiv cs.AI· Zhuoming Liu, Jinhong Lin, Kwan Man Cheng, Lin Zhang, Shayok Bagchi, Yin Li·· 13 小时前AI 评分31
DRIFT：用残差流适配器为视觉语言模型解码连续输出
DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models
AI 导读
DRIFT 是一个将预训练视觉语言模型适配到连续解码任务的通用框架，由提供粗略估计的基础预测器与基于流匹配的生成式精修模块组成，把生成建模问题转化为对强先验周围局部残差分布的建模。在视觉定位与机器人控制等感知和规划任务上，跨 MLLM、VLA、WAM 多种架构，DRIFT 一致优于回归类和生成类方案。
正文 · 原文
Abstract:Many modern vision-language models (VLMs) build on autoregressive decoding of discrete tokens. While text-based output interfaces enable scalable pretraining and strong zero-shot generalization across diverse tasks, they are poorly suited for problems that require precise continuous outputs, such as localizing temporal boundaries of events or generating robotic control actions. To address this challenge, we propose DRIFT, a general framework for adapting pretrained VLMs to continuous decoding tasks. DRIFT combines a base predictor, which provides a coarse estimate of the target output, with a generative refinement module based on flow matching that iteratively improves the prediction. This residual formulation transforms the generative modeling problem from learning a global output distribution to modeling a localized residual distribution around a strong prior, substantially simplifying optimization. We evaluate DRIFT on both perception and planning tasks, including visual grounding and robotic control. Across multiple tasks and architectures spanning MLLMs, VLAs, and WAMs, DRIFT consistently outperforms a strong set of regression- and generative-based solutions.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.05758 [cs.CV] |
| (or arXiv:2606.05758v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.05758 arXiv-issued DOI via DataCite |
Submission history
From: Zhuoming Liu [view email]
[v1]
Thu, 4 Jun 2026 06:37:10 UTC (3,544 KB)
[v2]
Tue, 27 Oct 2026 15:50:15 UTC (3,547 KB)
来源：arXiv cs.AI · arxiv.org