arXiv cs.AI· Dianye Huang, Jiamei Guo, Yue Zhou, Mingde Li, Yi Xiong, Wei Wang, Nassir Navab, Zhongliang Jiang·· 12 小时前AI 评分27
HAD-VLM：在医学视觉语言模型中用扩散预测保留数值连续性
Beyond Discrete Tokens: Preserving Numerical Continuity in Medical Vision-Language Models
AI 导读
HAD-VLM 提出混合自回归-扩散框架，在共享骨干内结合文本生成与连续测量预测，用扩散过程输出包含卡尺端点坐标与测量长度的五分量表示。
正文 · 原文
Abstract:Objective: To develop a unified vision-language model for prompt-conditioned quantitative ultrasound measurement that addresses the mismatch between continuous measurement targets and discrete token-based numerical prediction. Methods: This article presents HAD-VLM, a hybrid autoregressive-diffusion framework that combines autoregressive text generation with continuous measurement prediction within a shared backbone. Visual prompt tuning adapts the vision encoder to ultrasound images. A diffusion process predicts a five-component representation comprising caliper endpoint coordinates and measurement length. A post-processing strategy combining the proposed coordinate-length consistency mechanism with ensemble prediction is introduced to refine measurement estimates. We evaluate four echocardiographic measurement tasks on EchoNet-LVH, with additional assessments of medical question answering and X-ray vertebral measurement. Results: HAD-VLM achieves mean absolute errors of 1.36, 2.77, 3.05, and 1.16 mm for IVSd, LVIDd, LVIDs, and LVPWd, respectively, reducing errors by 21.4-37.6\% relative to the Vicuna-based autoregressive baseline. It also outperforms fine-tuned Qwen3-VL and direct MLP regression across all four tasks. Medical question answering performance is preserved on the evaluated dataset. For X-ray vertebral measurement, HAD-VLM reduces mean absolute error from 11.4 to 6.7 pixels. Conclusion: Integrating autoregressive language generation with diffusion-based continuous prediction improves quantitative medical image measurement while retaining prompt-conditioned task flexibility. Significance: HAD-VLM provides a unified approach to combining language interaction with anatomically grounded numerical prediction for quantitative medical image analysis.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32948 [cs.CV] |
| (or arXiv:2610.32948v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32948 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dianye Huang [view email]
[v1]
Fri, 23 Oct 2026 17:40:58 UTC (2,462 KB)
来源：arXiv cs.AI · arxiv.org