arXiv cs.CL· Minming Ye, Min Liu, Haoang Chi, Ciheng Wu, Naiyang Guan·· 2 天前AI 评分27
ECI2D：通过可执行因果干预蒸馏自生成上下文演示
From Observation to Intervention: Distilling Self-Generated In-Context Demonstrations via Executable Causal Intervention
AI 导读
研究者提出 ECI2D 方法，将语言理由抽象为可执行程序，对逻辑相似样本施加因果干预（do-operator），仅保留具备迁移不变性的自生成 ICL 演示。在数学、逻辑和常识推理基准上，ECI2D 持续优于基于观察的 ICL，且保留的演示数量远少，测试分布偏移时增益超过 7 分。
正文 · 原文
Abstract:Self-generated In-Context Learning (ICL) enables LLMs to synthesize their own few-shot demonstrations, often outperforming manual prompts. However, existing methods filter demonstrations by observation, assuming a rationale is high-quality if it leads to the correct output. We show such filtering suffers from \textit{observational equivalence}, whereby both faithful causal rationales and spurious post-hoc rationales can yield same outputs and are statistically indistinguishable from observation alone. Inspired by independent causal mechanisms, we argue that faithful rationales imply \textit{transfer invariance}, in which causal rationales should remain effective across logically similar samples. We instantiate this criterion as \textbf{E}xecutable \textbf{C}ausal \textbf{I}ntervention for \textbf{D}emonstrations \textbf{D}istillation (ECI2D), which abstracts language rationales into executable programs and applies the causal interventions ($do$-operator) on logically similar samples, retaining only those demonstrations that imply transfer invariance. Experiments on math, logical and commonsense reasoning benchmarks demonstrate that ECI2D consistently outperforms observation-based ICL while retaining far fewer demonstrations, with gains exceeding 7 points as the test distribution shifts.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21149 [cs.CL] |
| (or arXiv:2610.21149v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21149 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Minming Ye [view email]
[v1]
Thu, 15 Oct 2026 03:06:20 UTC (284 KB)
来源：arXiv cs.CL · arxiv.org