AIMO Interpretability Challenge：区分前沿数学模型的稳健推理与虚假推理

Authors:Michal Štefánik, Philipp Mondorf, Andreas Waldis, Michal Spiegel, Robert Belanec, Chuan Yang, Maria Matveev, Jan Mazak, Qianying Liu, Markus Schepers, Josef Kuchař, Marek Kadlčík, Adam Vawda-Oomerjee, Chaoran Liu, Simon Frieder, Barbara Plank, Fazl Barez, Pontus Stenetorp

View PDF HTML (experimental)

Abstract:We propose the AIMO Interpretability Challenge, a competition on distinguishing robust from spurious reasoning in frontier mathematical language models based on the models' internal mechanisms. The challenge is motivated by a central limitation of standard reasoning benchmarks: strong final-answer accuracy does not reveal whether a model relies on stable reasoning mechanisms or exploits brittle reasoning shortcuts. Building on AI Mathematical Olympiad (AIMO) problems and submissions, together with resources from the Fields Model Initiative, the competition will provide (1) newly-published olympiad-level math reasoning problems and their symbolic representations, allowing generation of novel functional variants, (2) access to frontier reasoning models, and (3) assessments of models' adversarial robustness on these problems. Participants will use these resources, along with our computing infrastructure support, to develop methods for identifying which models solve problems robustly. Our competition will also create a new, open robustness benchmark and baseline systems, aiming to provide a lasting foundation for standard benchmarking in mathematical reasoning and interpretability. Scientifically, the competition connects interpretability and generalization research around a central question in AI research: can we determine if, and to what extent, the decision-making of frontier AI models is generalizable and thus, reliable?

Submission history

From: Michal Štefánik [view email]
[v1] Wed, 15 Jul 2026 14:41:28 UTC (30 KB)
[v2] Tue, 20 Oct 2026 10:59:30 UTC (31 KB)