arXiv cs.CL· Yaowen Ye, Jacob Steinhardt, Sarah Schwettmann, Cassidy Laidlaw·· 12 小时前AI 评分55
研究团队自动生成带意外副作用的训练数据集，覆盖 8 种行为共 6575 个
Mass Producing Training Data with Surprising Side Effects
AI 导读
arXiv 论文提出自动发现带意外副作用的训练数据集（DSSE），要求数据能诱发目标行为、读起来像普通后训练数据、且让人类审阅者感到意外，并用 LLM 评判器将这三项标准量化。
正文 · 原文
Abstract:Training data can have unexpected side effects on language model behavior outside the training domain: insecure code data can induce broad misalignment, and outdated bird names can induce a 19th-century persona. Understanding such generalization is hard because only dozens of isolated examples exist, each constructed by hand or discovered by accident. We propose automatically discovering such datasets with surprising side effects (DSSEs). We define three objectives for DSSEs: they should (1) elicit a target behavior, (2) read like ordinary post-training data, and (3) be surprising to human auditors reading the data. We operationalize each with LLM judges, and search for high-scoring datasets via evolution, reinforcement learning, and subliminal learning approaches. After re-scoring and filtering with held-out metrics, our pipeline produces a collection of 6,575 DSSEs across 8 behaviors, many of which generalize across model families and sizes. We find our automated metrics track ML researchers' judgments, and frontier models outperform humans at predicting side effects. These datasets form a testbed for studying generalization and lay the groundwork for predicting training side effects before model deployment. Explore them at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.06324 [cs.CL] |
| (or arXiv:2611.06324v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06324 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yaowen Ye [view email]
[v1]
Wed, 4 Nov 2026 10:57:28 UTC (1,040 KB)
来源：arXiv cs.CL · arxiv.org