arXiv cs.CL· Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Volker Tresp·· 13 小时前AI 评分31
研究揭示文档级 LoRA 适配器的开卷分数主要衡量阅读行为
Mostly Reading: What Open-Book Scores Measure in Document-Level LoRA Adapters
AI 导读
在 Llama-3.1-8B-Instruct 上为 150 篇 Wikipedia 文章各训练一个 LoRA 适配器后发现，开卷分数主要衡量所有适配器共有的阅读行为：用另一文档训练的随机适配器可复现专家适配器相对基座模型 82% 的开卷优势，但在不提供原文的 cloze 探针上仅复现 2%。
正文 · 原文
Abstract:Single-document LoRA adapters have been proposed as units of knowledge to be merged and retrieved; yet open-book scoring, with the document in the prompt, can credit a document's own adapter (its specialist) for reading that document as readily as for holding it. Training one adapter for each of 150 Wikipedia articles on Llama-3.1-8B-Instruct, we find that open-book scores mostly measure a reading behaviour every adapter shares: a random adapter trained on another document recovers 82 % of the specialist's open-book advantage over the base model but only 2 % of its advantage on a cloze probe that asks for the same document's facts without providing the source text, a share we cannot distinguish from zero. Content does reach the weights, since the specialists beat the base model in cloze recall by 0.094, and the geometry of their updates follows the documents they read; yet, in a test of 30 queries, adapters chosen by weight-space proximity or text embedding are not distinguishable from random ones on either probe, although that test cannot exclude an advantage of about four fifths of what a query's own adapter gains over a random one. At 16 documents or more, correctly scaled merges recover 87 % to 89 % of the specialists' open-book advantage but only 12 % to 16 % of their cloze advantage, with intervals that run from below zero to at most 32 %, against 94 % for joint this http URL also report four silent errors, each of which produced a clean, wrong result.
| Comments: | 27 pages, 3 figures, 15 tables, under review at ACL ARR |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.31017 [cs.CL] |
| (or arXiv:2610.31017v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31017 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Martino M. L. Pulici [view email]
[v1]
Thu, 22 Oct 2026 09:52:29 UTC (59 KB)
来源：arXiv cs.CL · arxiv.org