arXiv cs.CL· Khoi-Nguyen Tran·· 6 天前AI 评分22
perturber：面向语义保持文本扰动的无状态服务
perturber: A Stateless Service for Semantics-Preserving Text Perturbations
AI 导读
perturber 是一个无状态服务和 Python 库，每次调用只施加一个语义保持扰动，来自覆盖 11 个家族的 68 个扰动的一手目录。它基于单一原语，组合交由调用方链式调用，因此服务与库的输出天然一致。该工作评估工具本身的正确性、可复现性与成本，并逐扰动测量语义保持性，而非假设其成立。
正文 · 原文
Abstract:Large language model (LLM) evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning-preserving variants routinely move model answers. Studying this systematically requires applying perturbations one at a time, reproducibly, and identically wherever they run, yet existing tools couple them to a benchmarking stack rather than packaging them as a reusable transform layer. We present perturber, a stateless service and Python library that applies exactly one semantics-preserving perturbation per call, drawn from a first-party catalog of 68 perturbations across 11 families. It turns on a single primitive, with composition left to the caller chaining calls, so the service and the library produce identical output by construction. We evaluate the tool itself, its correctness, reproducibility, and cost, rather than any model's robustness, measuring meaning-preservation per perturbation rather than assuming it. The demonstration adds a live playground, an introspectable catalog for benchmark selection, and a one-command reproducible evaluation.
| Comments: | 14 pages (6 main + references and appendix), 5 figures, 13 tables. Code: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.12667 [cs.CL] |
| (or arXiv:2610.12667v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12667 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Khoi-Nguyen Tran [view email]
[v1]
Thu, 8 Oct 2026 19:44:05 UTC (153 KB)
来源：arXiv cs.CL · arxiv.org