arXiv cs.CL· Jinjun Peng, Magnus Saebo, Tianjun Zhong, Junfeng Yang, Shi-Xiong Zhang, Baishakhi Ray, Simin Chen, Yangruibo Ding·· 3 天前AI 评分36
SWE-Spot：用仓库中心学习打造小型代码库专家模型
SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning
AI 导读
针对小型开源权重模型缺乏对陌生代码库泛化能力的问题，研究者提出仓库中心学习（RCL），以单仓库纵向深度替代跨仓库的任务中心学习（TCL），并据此训练出 SWE-Spot-4B 系列仓库专家模型。
正文 · 原文
Abstract:The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs). However, they suffer from a fundamental capability gap: unlike frontier large models, they lack the inference-time strong generalization to handle complicated, unfamiliar codebases. We identify that the prevailing Task-Centric Learning (TCL) approach, which scales exposure across disparate repositories, fails to address this limitation. In response, we propose Repository-Centric Learning (RCL), a shift that prioritizes vertical, single-repository depth over horizontal, single-task breadth for SLM training. To accomplish this depth, we design a four-unit Repository-Centric Experience, transforming static codebases into interactive learning signals, to train SWE-Spot-4B, a family of highly compact models built as repo-specialized experts. Under controlled comparisons with the same teacher and data budget, RCL consistently outperforms TCL across tasks, teachers, model scales, and programming languages. On seven target repositories, SWE-Spot-4B experts match or exceed existing open-weight models up to $8\times$ larger (e.g., CWM by Meta, Qwen3-Coder-30B) and surpass or match efficiency-focused commercial models (e.g., GPT-4.1-mini, GPT-5-nano) across multiple SWE tasks. Further analysis reveals that RCL yields higher training and inference efficiency, deep knowledge acquisition, and can scale to repository-cluster experts. Our work emphasizes that for building efficient intelligence, repository mastery is a distinct, necessary, and practical dimension that complements general coding capability.
| Comments: | substantially revised |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Software Engineering (cs.SE) |
| Cite as: | arXiv:2601.21649 [cs.LG] |
| (or arXiv:2601.21649v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2601.21649 arXiv-issued DOI via DataCite |
Submission history
From: Jinjun Peng [view email]
[v1]
Thu, 29 Jan 2026 12:49:25 UTC (331 KB)
[v2]
Tue, 13 Oct 2026 18:52:58 UTC (312 KB)
来源：arXiv cs.CL · arxiv.org