arXiv cs.CL· Louis Robinson·· 12 小时前AI 评分34
Causal Edit Serialization：在因果 token 流中实现局部编辑的框架
Causal Edit Serialization
AI 导读
Causal Edit Serialization（CES）框架让自回归语言模型在同一因果 token 流内完成插入、删除、替换、复制和移动等局部编辑，通过特殊指针 token 配合预测的源地址实现定位。
正文 · 原文
Abstract:Standard autoregressive language models generate text sequentially, but revision requires referring back to tokens in the input context or those already generated. Causal Edit Serialization (CES) is introduced as a framework for expressing such revisions within the same causal token stream. Special pointer tokens are paired with predicted source addresses, allowing insertion, deletion, replacement, copying and movement to be localized explicitly. A lightweight localization head and dual-index rotary position encoding support both address prediction and conditioning on previously emitted pointers, while retaining a standard causal attention mask. A pretraining procedure combines ordinary language modeling with serialized repairs derived from random corruptions, model-generated edits and repeated text. Across model scales and editing baselines, CES models demonstrate complete applied edits with modestly higher clean-text bits per byte than clean autoregressive models. Supervised and reinforcement-learning post-training provide a grammatical-error-correction proof of concept, producing more beneficial than harmful changes under reference-blind model judging. Correct repair programs also improve subsequent text prediction while the original context and its cached states are retained. CES provides a common interface for generation and localized revision, with potential extensions to iterative refinement and additional pointer-addressed transformations.
| Comments: | 37 pages, including appendices and checklist. Code: this https URL ; data and checkpoints: this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.22820 [cs.CL] |
| (or arXiv:2610.22820v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22820 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Louis Robinson [view email]
[v1]
Fri, 16 Oct 2026 01:55:26 UTC (516 KB)
来源：arXiv cs.CL · arxiv.org