Online Learning with LLM Experts from Limited Feedback
Paper proposes bandit algorithms for adaptively routing prompts to LLM experts, minimizing regret under limited feedback budgets.
The paper formulates adaptive prompt routing to K LLM experts as a contextual bandit problem with d prompt features over T rounds. Proposed algorithms strategically select actions and observe rewards, achieving O(dT/m) regret in the full-information setting and O(dTK/m) in the bandit setting, where m is the feedback budget. Experiments demonstrate efficient learning of high-quality routing strategies across diverse LLMs from limited feedback.
- Formulates LLM prompt routing as contextual bandit with K experts, d features, T rounds
- Achieves O(dT/m) full-information regret and O(dTK/m) bandit-setting regret
- Feedback budget m (m much less than T) constrains reward observations
- Experiments show efficient routing strategies learned across diverse LLMs
Full article110 words · extracted from huggingface.co · click to collapse
We study adaptive routing of prompts to large language model (LLM) experts to maximize response quality in an online setting with limited feedback. We formulate it as a bandit problem with K actions that represent experts and d features that encode prompts, over a horizon of T rounds. We propose algorithms that strategically select and observe rewards to minimize regret. In the full-information setting, we achieve a regret of O(d T / m), while in the bandit setting we achieve O(d T K / m), where m ll T is a budget on feedback. Our experiments show that we efficiently learn high-quality routing strategies across diverse LLMs from limited feedback.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.05820