综述：机制可解释性如何服务于 AI 安全
Understanding AI from the Inside: The State of Mechanistic Interpretability for AI Safety
这篇综述围绕“理解模型内部机制是否真能让模型更安全”这一问题，系统梳理了机制可解释性研究。文章介绍特征、多义性、Transformer 计算与电路等基础概念，综述特征提取、电路发现与因果验证、用自然语言解释中间表示、学习动态和训练数据归因等方法，并提出一个统一框架来评估机制可解释性主张，按目标、假设与权衡划分六种研究取向。
Authors:Fazl Barez, Tung-Yu Wu, Hunar Batra, Gabriele Dominici, Florent Draye, Yanai Elazar, Damiano Fornasiere, Antía García, Abir Harrasse, Riley Harris, Sara Hooker, Michael Lan, Belinda Z. Li, Aaron Maiwald, Narmeen Fatimah Oozeer, Naomi Saphra, Or David Shafran, Adi Simhi, Robert Trager, Shriyash Kaustubh Upadhyay, Ziyu Yao, Yu Zhao, Shay B. Cohen, Mor Geva, Yoshua Bengio, Stella Biderman, Hassan Sajjad
Abstract:Neural networks are likely to be among the most consequential artefacts in human history. As these systems become more capable and autonomous, their potential to cause societal harm also grows, making AI safety one of the defining challenges of our time. Yet we continue to build increasingly powerful systems without fully understanding how they work, how they will behave in unfamiliar situations, or how to reliably control them. Mechanistic interpretability offers a path towards understanding these systems by studying their internal representations and computations, with the aim of evaluating, predicting, and controlling behaviour. We survey mechanistic interpretability for AI safety in order to answer the question: Does understanding how models work actually make them safer? We place mechanistic interpretability in the broader context of interpretability research and compare its mechanistic evidence with behavioural evaluation. We introduce the foundations of mechanistic interpretability, including features, polysemanticity, transformer computation, and circuits, and survey methods for feature extraction and analysis, circuit discovery and causal verification, explaining intermediate representations in natural language, learning dynamics, and training-data attribution. Beyond surveying existing methods, we propose a unified framework for evaluating mechanistic interpretability claims and characterise six research orientations by their goals, assumptions, and trade-offs. Finally, we examine how mechanistic understanding can support AI safety, including knowledge editing, steering, and monitoring, and assess its limitations, failure modes, open problems, and relationship to behavioural safety measures. We also release a repository of code notebooks with worked examples that reproduce key findings from surveyed methods, allowing readers to build intuition through hands-on experimentation.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.08438 [cs.CL] |
| (or arXiv:2611.08438v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08438 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tungyu Wu [view email]
[v1]
Thu, 5 Nov 2026 18:30:54 UTC (3,892 KB)
来源：arXiv cs.CL · arxiv.org