LLM 路由与级联的统一方法：cascade routing 框架

View PDF HTML (experimental)

Abstract:The availability of a wide range of large language models (LLMs) embedded in various agentic systems has significantly increased the potential of model selection strategies to improve the cost-performance tradeoff. Existing strategies involve either routing, where a single model is chosen per query, or cascading, which sequentially runs increasingly larger models until a satisfactory answer is found. However, current approaches face three key limitations: they (1) lack formal proofs of optimality, (2) fail to identify the conditions under which these strategies are most effective to improve the cost-performance tradeoff, and (3) are unable to combine both paradigms for further improvements. To address these issues, we first derive a novel optimal strategy for cascading and prove the optimality of an existing routing strategy. Further, we propose cascade routing, a unified framework that integrates routing and cascading into a theoretically optimal strategy. Through our analysis, we identify good quality estimators as the critical factor for the success of model selection paradigms. Finally, in our experiments, we show that cascade routing consistently outperforms the individual approaches by a large margin and we analyze quality estimators to determine when routing and/or cascading are useful paradigms for model selection.

Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2410.10347 [cs.CL]
	(or arXiv:2410.10347v5 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2410.10347 arXiv-issued DOI via DataCite

Submission history

From: Jasper Dekoninck [view email]
[v1] Mon, 14 Oct 2024 10:00:49 UTC (263 KB)
[v2] Fri, 14 Feb 2025 13:13:33 UTC (801 KB)
[v3] Thu, 22 May 2025 12:34:14 UTC (994 KB)
[v4] Mon, 12 Oct 2026 09:00:03 UTC (731 KB)
[v5] Mon, 26 Oct 2026 22:53:14 UTC (731 KB)