arXiv cs.CL· Yang Liu, Ran Zou, Wanrong Zhu·· 12 小时前AI 评分24
Source-balanced Capped Fisher：语言模型用户推断的有界证据聚合方法
Source-balanced Capped Fisher: Bounded Evidence Aggregation for User Inference in Language Models
AI 导读
研究者提出 source-balanced capped Fisher（SCF），一种利用已知未参与来源评估文档分数异常程度的聚合方法，将参考尾部频率转为负对数证据、截断单文档贡献并按来源平均，以抑制孤立极端分数的影响。
正文 · 原文
Abstract:User inference asks whether a source contributed documents to a language-model update, even when the auditor observes only fresh documents from that source. Document-level scores can provide evidence of participation, but combining this evidence is challenging: documents differ in baseline predictability and in how strongly they reflect information learned from other documents of the same source. We propose source-balanced capped Fisher (SCF), an aggregation method that uses known nonparticipating sources to assess how unusual each document score is. It transforms the resulting reference-tail frequencies into negative-log evidence, caps individual contributions, and averages them within each source. This limits the influence of isolated extreme scores while retaining evidence across multiple documents. We complete the inference procedure with a decision rule that tests source participation using a p-value calibrated against a separate set of known nonparticipating sources. The rule provides finite-sample Type-I error control under source exchangeability. Experiments across four real-world datasets show that SCF achieves the highest AUROC among the evaluated aggregation methods in four domains. With the calibration-based decision rule, SCF delivers consistently competitive detection power across the tested settings, achieving the highest power in many cases, while empirical false-positive rates remain generally close to or below the nominal levels.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.07228 [cs.CL] |
| (or arXiv:2611.07228v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07228 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ran Zou [view email]
[v1]
Wed, 4 Nov 2026 23:33:10 UTC (186 KB)
来源：arXiv cs.CL · arxiv.org