arXiv cs.CL· Haiyan Zhao, Zirui He, Yiming Tang, Fan Yang, Ali Payani, Dianbo Liu, Mengnan Du·· 12 小时前AI 评分44
Rep2Text：从单个 LLM Token 表示中解码完整文本
Rep2Text: Decoding Full Text from a Single LLM Token Representation
AI 导读
Rep2Text 能从单个隐藏表示中重建原始文本，对约 16 词的短文本可恢复约一半原词并保留整体语义。文本变长后精确措辞更难还原，但主要观点仍可恢复。该结果揭示了语言模型内部存储的信息量，并提示隐藏表示可能泄露部分原始输入的隐私风险。
正文 · 原文
Abstract:Large language models represent text as hidden numerical patterns, but what information these representations actually contain remains a mystery. Can we recover the original text from just one of them? In this paper, we introduce Rep2Text, a method designed to answer this question by reconstructing text from a single hidden representation. We find that these representations preserve much more information than we might expect. For short texts of about 16 words, Rep2Text can recover roughly half of the original words while preserving their overall meaning. As the text gets longer, recovering the exact wording becomes harder, but the main ideas can still be recovered. These findings offer a new way to understand what language models store internally and highlight potential privacy risks, as hidden representations may reveal parts of the original input.
| Comments: | 28 pages, 11 figures, 8 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2511.06571 [cs.CL] |
| (or arXiv:2511.06571v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2511.06571 arXiv-issued DOI via DataCite |
Submission history
From: Haiyan Zhao [view email]
[v1]
Sun, 9 Nov 2025 23:18:36 UTC (359 KB)
[v2]
Fri, 20 Mar 2026 13:53:32 UTC (327 KB)
[v3]
Fri, 8 May 2026 15:19:37 UTC (1,741 KB)
[v4]
Fri, 16 Oct 2026 05:27:30 UTC (1,801 KB)
来源：arXiv cs.CL · arxiv.org