arXiv cs.CL· Vladimir Beskorovainyi·· 12 小时前AI 评分34
面向外部 LLM 的可逆假名化：开放双语基准与引擎
Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine
AI 导读
研究者发布了一个面向俄语和英式英语的开放基准，评测五款现有开源工具与一个正则基线，并基于其中最强的两款构建了可逆假名化引擎。在每语言 1000 篇的留出集上，最佳现有工具仍有 61%（俄语）和 58%（英式英语）的文档泄露完整敏感值，该引擎仅 0.3% 和 0.2%，且两种语言均满足全部检测阈值。
正文
Abstract:Professionals send working documents to external large language models. A common safeguard is reversible pseudonymisation: personal data are replaced with placeholders before the request and restored locally in the answer. We present an open benchmark of five existing open tools and a regular-expression baseline on Russian and UK English, and an open engine built on the two strongest tools. The benchmark excludes each tool's own authors' dataset from its verdict, compares the engine with each tool on identical datasets for detection, reports masking precision, and tests on a held-out set of synthetic documents whose files, hashes, decision rules, run plan and stopping rule were published before any run. Two local LLMs test whether placeholders survive. All evaluation data are synthetic, and the benchmark was run by the author of the engine; code, data, predictions and raw answers are released. On external datasets no existing tool meets all predefined thresholds: in UK English the identifier and leak thresholds are not met, in Russian the identifier threshold is borderline. On the last held-out set of 1,000 documents per language, the best existing tools leave a whole value visible in 61% (Russian) and 58% (UK) of documents. On these documents, from the generator it was developed on, the engine does so in 0.3% [0.1; 0.9] and 0.2% [0.1; 0.7], leaves any part of a value visible in 0.8% and 0.2%, and meets all detection thresholds in both languages. In UK English it also restores at least 99% of its placeholders after 760 answers. In Russian it restores only 94.6% [93.6; 95.6] of name placeholders, against a 99% threshold, mainly because the inflection it borrows from pii-guard produces wrong surname forms. The engine was developed over six earlier held-out sets from the same generator; real documents were not tested.
| Comments: | 22 pages, 9 tables. Code, data generator, held-out sets, pre-registrations, predictions and raw LLM answers: this https URL (release v0.5.0, DOI https://doi.org/10.5281/zenodo.23624098) |
| Subjects: | Computation and Language (cs.CL); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2610.34012 [cs.CL] |
| (or arXiv:2610.34012v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34012 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vladimir Beskorovainyi [view email]
[v1]
Sun, 25 Oct 2026 02:50:43 UTC (29 KB)
来源：arXiv cs.CL · arxiv.org