VB-EGE：面向多目标 LLM 竞技场的帕累托前沿自适应采样算法

View PDF HTML (experimental)

Abstract:The evaluation of large language models (LLMs) is inherently multi-objective, as different models may offer different tradeoffs across performance dimensions such as math, code, and writing. Because such evaluations are often based on costly and time-consuming human feedback, it is important to draw reliable conclusions using as few comparisons as possible. In this paper, we present Vector-Borda Empirical-Gap Elimination (VB-EGE), a fixed-confidence adaptive sampling algorithm for efficiently identifying models on the Pareto frontier in arena-style evaluation, where each comparison evaluates a pair of models on a prompt with respect to a selected performance dimension. VB-EGE progressively eliminates models from an active set once sufficient statistical evidence has been accumulated to certify their Pareto status. Under a multi-objective Bradley-Terry model, we show that VB-EGE returns the correct Pareto frontier with high probability and derive an instance-dependent upper bound on its sample complexity. We further establish a worst-case lower bound showing that VB-EGE is minimax-optimal up to logarithmic factors. Simulations and experiments on three real-world Arena datasets (Arena, SearchArena, and VisionArena) demonstrate VB-EGE's empirical effectiveness.

Comments:	51 pages, including references and appendices
Subjects:	Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Applications (stat.AP); Other Statistics (stat.OT)
Cite as:	arXiv:2610.27447 [stat.ML]
	(or arXiv:2610.27447v1 [stat.ML] for this version)
	https://doi.org/10.48550/arXiv.2610.27447 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Liang Twist Shan [view email]
[v1] Tue, 20 Oct 2026 00:50:18 UTC (1,705 KB)