arXiv cs.AI· Naiyu Yin, Dennis Wei, Tian Gao, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Yue Yu·· 6 天前AI 评分22
CircuitLasso：面向大语言模型可解释性的可扩展电路学习方法
Scalable Circuit Learning for Interpreting Large Language Models
AI 导读
研究者提出 CircuitLasso，一种基于稀疏线性回归的可扩展电路学习方法，用于解释大语言模型。它在基准数据上的电路结构准确率与现有基于干预的 SOTA 方法相当，但计算成本大幅降低，并能高效揭示 SAE 特征之间的传播关系。该方法已被 EMNLP 2026 主会接收。
正文 · 原文
Abstract:A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit learning methods computationally prohibitive. We propose CircuitLasso, a scalable circuit learning approach based on sparse linear regression. CircuitLasso recovers circuits whose structural accuracy matches that of state-of-the-art intervention-based methods on the benchmark data, at a fraction of the computational cost. For interpretability, CircuitLasso efficiently uncovers relationships among SAE features, showing how human-interpretable semantic features propagate through the model and influence its predictions. Finally, we validate the utility of our learned circuits by leveraging their insights to achieve comparable performance at substantially lower cost on a domain-generalization task.
| Comments: | Accepted to the Main Conference of EMNLP 2026 (The 2026 Conference on Empirical Methods in Natural Language Processing) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.16939 [cs.LG] |
| (or arXiv:2606.16939v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2606.16939 arXiv-issued DOI via DataCite |
Submission history
From: Naiyu Yin [view email]
[v1]
Mon, 15 Jun 2026 16:40:43 UTC (996 KB)
[v2]
Fri, 9 Oct 2026 01:39:55 UTC (983 KB)
来源：arXiv cs.AI · arxiv.org