Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing
RouteFM pretrains a frozen LLM router that generalizes to new models using behavioral context alone.
RouteFM treats LLM routing as a foundation-model problem, characterizing anonymous candidate models from behavioral context instead of binding decisions to fixed identities. Episodic pretraining across heterogeneous environments produces a frozen router that transfers across domains, modalities, candidate pools, and context budgets. On MMR-Bench, excluded from pretraining, it outperforms the strongest baseline by 2.23 quality points with eight observations per candidate. Code is released at LAMDA-Model-Reuse/RouteFM.
- RouteFM learns reusable routing from behavioral context, not fixed model identities.
- A frozen router adapts to new pools through context alone.
- On held-out MMR-Bench it beats the strongest baseline by 2.23 quality points.
- Largest gains occur with only eight observations per candidate.
- Code is public at LAMDA-Model-Reuse/RouteFM.
From the original article
1 min read at huggingface.co
Large language model (LLM) routing aims to assign each query to the most suitable model from a heterogeneous candidate pool, improving the quality--efficiency trade-off of LLM inference. Existing routers are typically learned through local fitting: a router is optimized for a particular query workload and candidate pool, and often requires additional supervision or retraining as the routing environment changes.
By Guannan Lai, Han-Jia Ye · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.