arXiv cs.AI· Yunqin Zhu, Yao Xie·· 14 小时前AI 评分22
Merge and Branch：面向推荐的高效树结构 LLM 监督微调方法
Merge and Branch: Efficient Tree-Structured Supervised Fine-Tuning of LLMs for Recommendation
AI 导读
Merge and Branch（M&B）通过将每条历史物品的 SID 与文本压缩为少量学习 token，并用压缩后的历史同时预测多个目标 SID，把每个样本组织成浅而宽的树结构，从而降低 LLM 推荐微调的输入成本。
正文 · 原文
Abstract:LLM-based generative recommendation reads long, text-rich interaction histories but generates only short semantic identifiers (SIDs). This asymmetry makes supervised fine-tuning inefficient: encoding the history is expensive, yet each history usually provides supervision for only one target. We present Merge and Branch (M&B) to reduce input cost and learn more from each encoded history. Merge compresses each historical item's SID and text into a small number of learned tokens. Branch uses the compressed history to predict multiple target SIDs. To share history computation while keeping these targets independent, M&B organizes each example as a shallow, wide tree. We develop tree FlashAttention to execute this layout efficiently, processing the shared history once and propagating gradients from every branch for end-to-end training. A tree-structured key/value cache extends history sharing to constrained decoding. Experiments on public benchmarks show that M&B outperforms strong recommendation baselines while substantially improving training and decoding efficiency.
| Comments: | 12 pages, 8 figures |
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.37973 [cs.IR] |
| (or arXiv:2610.37973v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37973 arXiv-issued DOI via DataCite |
Submission history
From: Yunqin Zhu [view email]
[v1]
Tue, 27 Oct 2026 21:10:11 UTC (741 KB)
来源：arXiv cs.AI · arxiv.org