arXiv cs.AI· Mingning Guo, Mengwei Wu, Shaoxian Li, Haifeng Li, Chao Tao·· 1 天前AI 评分22
AerialVP：用更强任务提示词引导视觉语言模型提升无人机图像感知
Towards Accurate UAV Image Perception: Guiding Vision-Language Models with Stronger Task Prompts
AI 导读
AerialVP 是一个任务自适应、免训练的智能体，通过提示词增强而非参数优化让视觉语言模型适配无人机图像感知任务。它由基于 LLM 的 Task Engine 协调提示词分析、视觉信息提取与提示词融合，按原始提示词的具体程度选择性注入图像语义、空间位置和物体关系线索，后端 VLM 保持不变。
正文 · 原文
Abstract:UAV image perception with VLMs aims to identify and localize targets based on semantic and spatial task prompts. However, small, densely distributed objects and cluttered backgrounds hinder these tasks, particularly when concise prompts provide insufficient guidance. Although task-specific fine-tuning improves image-text alignment, it requires large-scale annotations and model-specific updates, increasing adaptation costs across models and tasks. To address this, we propose AerialVP, a task-adaptive, training-free agent that adapts VLMs through prompt enhancement rather than parameter optimization. AerialVP dynamically adapts its enhancement strategy to the original prompt's specificity, selectively incorporating image-derived semantics, spatial positions, and object relationships. These cues provide discriminative descriptions and spatial constraints, leveraging VLMs' pretrained alignment and reasoning capabilities to improve image-text alignment and reduce target ambiguity. Specifically, an LLM-based Task Engine coordinates modular tools for prompt analysis, visual information extraction, and prompt fusion. The enhanced prompt and original image are then provided to the unchanged backend VLM for prediction. This prompt-centric strategy enables training-free adaptation across diverse open-source and proprietary VLMs. For systematic evaluation, we introduce AerialSense, a unified benchmark covering Aerial Visual Grounding, Aerial Visual Reasoning, and Aerial Visual Question Answering across diverse UAV scenes and imaging conditions. Experiments demonstrate consistent improvements across all three tasks and evaluated VLMs. AerialVP also achieves competitive performance against LoRA-based instruction fine-tuning, validating prompt-centric adaptation. Our work is available at this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2512.07302 [cs.CV] |
| (or arXiv:2512.07302v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2512.07302 arXiv-issued DOI via DataCite |
Submission history
From: Mingning Guo [view email]
[v1]
Mon, 8 Dec 2025 08:44:57 UTC (35,366 KB)
[v2]
Wed, 11 Nov 2026 10:13:40 UTC (35,124 KB)
来源：arXiv cs.AI · arxiv.org