arXiv cs.AI· Adrians Skapars, Edoardo Manino, Youcheng Sun, Lucas C. Cordeiro·· 12 小时前AI 评分42
SODA 算法可从输出逆向还原 LLM 输入文本
GPT, But Backwards: Exactly Inverting Language Model Outputs
AI 导读
研究者提出白盒搜索算法 SODA，可从输出 logits 重建 LLM 的文本输入，在最长 17 个 token 的自然序列上实现零误报，精确匹配重建效果较此前最强方法提升 4 倍。
正文 · 原文
Abstract:Recent work shows that distinct inputs produce distinct hidden representations in LLMs, so the input can in principle be recovered from them. Yet, no existing method can reliably do so from the model's output text or logits alone. We introduce Sparse One-hot Discrete Adam (SODA), a white-box search-based algorithm that reconstructs textual inputs from partial knowledge of the output. Given output logits, SODA recovers natural sequences of up to 17 tokens with zero false positives and achieves a 4x improvement in exact-match reconstruction over the strongest prior method. We further show that input length and vocabulary size matter more for inversion difficulty than parameter count, with SODA inverting models from 33M to 3B parameters. Finally, we demonstrate proof-of-concept security applications: SODA recovers 6 backdoor triggers from a poisoned LLM in under 1000 iterations, and 139 tokens of personally identifiable information from synthetic user-chatbot interactions. Our code is available at this https URL.
| Comments: | 8 pages, ICML 2025 Workshop on Reliable and Responsible Foundation Models |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2507.01693 [cs.LG] |
| (or arXiv:2507.01693v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2507.01693 arXiv-issued DOI via DataCite |
Submission history
From: Adrians Skapars [view email]
[v1]
Wed, 2 Jul 2025 13:20:30 UTC (1,269 KB)
[v2]
Mon, 10 Nov 2025 15:40:58 UTC (1,418 KB)
[v3]
Wed, 14 Oct 2026 23:49:34 UTC (1,625 KB)
来源：arXiv cs.AI · arxiv.org