AItonomy Foundation 发布 Science's Last Exam 科研智能体基准

Rohan Paul· @rohanpaul_ai · X·2026-11-06 03:04· 1 小时前AI 评分49

AI 导读

美国非营利机构 AItonomy Foundation 推出 Science's Last Exam（SLE）基准，用专家预先封存、对智能体隐藏的检查项，评测研究智能体可打开并复跑的研究产物。

正文 · 原文

Lean gave AI math a grader nobody can argue with, and empirical science is now getting its own version of that idea.

Science’s Last Exam (SLE), a benchmark, grades research agents on artifacts that can be opened and re-run, with checks that domain experts seal when each task is built.

Led by the US nonprofit AItonomy Foundation, the benchmark spans 113 tasks in 9 disciplines and grades submissions with checks hidden from agents.

Most AI science tests ask questions with known answers; SLE hands an AI agent a research goal, data and lab software, then checks its finished work with tests the agent never sees.

It scores AI hypotheses against studies published after the model’s training cutoff, so memory cannot help. DeepSeek V4.1 Flash led, with 21.28% of its hypotheses anticipating later research.

引用AItonomy Foundation@aitonomyfound

I’m incredibly proud to share what we’ve been building at the AItonomy Foundation: Science’s Last Exam (SLE). 🧪✨ Can AI do science? Instead of Q&A, we give agents actual research goals and real-world tools (lab data, software), then grade them against hidden expert checks. Deeply grateful to the brilliant minds at 34 institutions (Harvard, MIT, Stanford, etc.) who helped craft 113 tasks across 9 disciplines. ❤️ The results? Science is hard. Of 8 frontier models, the best scored 37.5/100. On true end-to-end research? All scored 0. We have an exciting journey ahead! 🚀 (Claude Opus 5.5 & GPT-6 Astra coming soon—just clearing bio safeguards). Everything opens Nov 6. 🗓️ 🌐 https://sciences-last-exam.org 💻 https://github.com/Gen-Verse/Science-Last-Exam 🤗 https://huggingface.co/datasets/Gen-Verse/Science-Last-Exam Thread below on how the exam works and where models break. 👇 (1/14) #AGI #AIforScience #AIAgents #SciencesLastExam #OpenAI #Anthropic

在 X 查看被引用的帖子

来源：Rohan Paul · x.com