arXiv cs.AI· Junnuo Wang, Weihan Xu, Hao-Wen Dong·· 12 小时前AI 评分31
Clips2Story：用多模态检索嵌入生成实现免训练视频分镜与编辑
Clips2Story: Training-Free Video Storyboarding and Editing Using Multimodal Retrieval-Embedded Generation
AI 导读
Clips2Story 是一个免训练、基于 LLM 的视频分镜流水线，可从无序视频片段池中按用户关键词生成短视频。它先为每个片段提取文本化多模态描述符并检索相关片段，再由 LLM 挑选排序成带连接旁白的可编辑分镜，最后组装片段并检索旁白配图合成视频。团队用 100 个长视频构建合成评测集，人工评估显示其在叙事连贯性、关键词对齐和整体真实感上优于强基线。
正文 · 原文
Abstract:Video editors often work with large collections of fragmented footage and must select and organize relevant clips into a coherent story. However, many existing video editing and summarization methods assume a temporally ordered source video and aim to preserve its original narrative and temporal structure. In this paper, we introduce Clips2Story, a training-free, LLM-based pipeline that enables users to create keyword-specified short videos from a pool of video clips. Given a pool of clips and a user-specified keyword, we first extract text-based multimodal descriptors for each clip and retrieve the clips most relevant to the keyword. We then provide the textual descriptors of these candidate clips to an LLM, which selects and arranges them into an editable storyboard with connective narrations. Finally, we assemble the selected clips and retrieve accompanying visuals for the narrations to compose the output video. To evaluate the proposed pipeline, we construct a synthetic evaluation dataset from 100 long-form videos across five genres by segmenting each video into short clips using a scene-change detection model and shuffling the resulting clips to remove their temporal order. Through objective evaluation and qualitative examples, we demonstrate that Clips2Story can effectively retrieve keyword-relevant clips and compose coherent storyboards based on user-specified keywords. Human evaluation further shows that Clips2Story outperforms a strong baseline in terms of narrative coherence, keyword alignment, and overall realism.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM) |
| Cite as: | arXiv:2611.05992 [cs.CV] |
| (or arXiv:2611.05992v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05992 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Junnuo Wang [view email]
[v1]
Wed, 4 Nov 2026 05:37:16 UTC (4,756 KB)
来源：arXiv cs.AI · arxiv.org