arXiv cs.AI· Jiayi Chen, Hui Li, Yujiong Shen, Qiyuan Peng, Yanxin Li, Chenhao Huang, Shaofan Liu, Guoqiang Zhang, Junjie Ye, Shihan Dou, Ming Zhang, Maxm Pan, Tao Gui, Qi Zhang, Xuanjing Huang·· 13 小时前AI 评分31
TIDE：基于模型行为的树诱导细粒度诊断评估
TIDE: Tree-Induced Fine-Grained Diagnostic Evaluation from Model Behaviors
AI 导读
研究者提出 TIDE（Tree-Induced Diagnostic Evaluation），将诊断评估重构为行为驱动的决策树归纳，从基准样本与模型得分递归构建分层能力树，由 LLM 裁判提出候选划分、以基于行为的信息增益选择划分。
正文 · 原文
Authors:Jiayi Chen, Hui Li, Yujiong Shen, Qiyuan Peng, Yanxin Li, Chenhao Huang, Shaofan Liu, Guoqiang Zhang, Junjie Ye, Shihan Dou, Ming Zhang, Maxm Pan, Tao Gui, Qi Zhang, Xuanjing Huang
Abstract:Benchmark performance is the dominant standard for comparing large language models (LLMs), typically summarized by a single aggregate score, such as accuracy, pass rate, or rubric score. This aggregate score provides limited information about a model's underlying capabilities, yet understanding these capabilities is crucial for model development and evaluation. To this end, we propose Tree-Induced Diagnostic Evaluation (TIDE), which recasts diagnostic evaluation as behavior-driven decision-tree induction to perform fine-grained capability diagnosis. TIDE recursively builds a hierarchical capability tree from benchmark samples and model scores: at each node, an LLM judge proposes candidate splits and performs sample-to-group classification, and behavior-based information gain selects among them. We apply TIDE to five models across four benchmarks to demonstrate its effectiveness, and find fine-grained capability contrasts invisible at the aggregate level: Claude Opus 4.6 achieves 100% on microeconomic market reasoning where GPT-5.4 scores 83.3%, but trails GPT-5.4 on contract law reasoning with 77.8% versus 88.9%. We hope TIDE can be extended to past and future benchmarks so that the rich behavioral signal in evaluation results is fully leveraged for understanding model capabilities.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.29985 [cs.AI] |
| (or arXiv:2610.29985v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29985 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiayi Chen [view email]
[v1]
Wed, 21 Oct 2026 16:27:48 UTC (637 KB)
来源：arXiv cs.AI · arxiv.org