SeLMRoute: Probabilistic Semantic Evidence for Large Language Model Routing
SeLMRoute routes queries with probabilistic semantic evidence and beats the strongest fixed model on LLMRouterBench.
SeLMRoute separates candidate-independent semantic judgments about a query from performance estimation and deployment objectives. A decision model answers interpretable questions, such as reasoning needs and external-knowledge use, and stores each judgment as a probability distribution for a lightweight supervised router. On LLMRouterBench, covering 15 datasets, 20 models, and 11,481 queries, it reaches 72.08% average accuracy versus 69.23% for the strongest fixed candidate, and 72.64% in grouped five-fold evaluation. In a separate 13-model performance-cost setting it improves every grouped split, with a mean PerfGain of 2.66%.
- Semantic evidence is candidate-independent probabilities over interpretable query questions.
- Routing objectives apply after performance estimation, including cost-aware choices.
- LLMRouterBench accuracy is 72.08% versus 69.23% for the best fixed model.
- A 13-model cost setting shows 2.66% mean PerfGain across five splits.
From the original article
1 min read at huggingface.co
Large language model (LLM) routing aims to select the most suitable model for each incoming query. Most existing routers learn this decision directly from query embeddings, model representations, preference data, or clusters of similar examples. Such approaches can be effective, yet the representation used for routing rarely states what a query actually requires.
By Vasilis Perifanis, Nikolaos Pavlidis, Symeon Symeonidis · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.