Online Learning with LLM Experts from Limited Feedback
Paper proposes bandit algorithms for adaptively routing prompts to LLM experts, minimizing regret under limited feedback budgets.
The paper formulates adaptive prompt routing to K LLM experts as a contextual bandit problem with d prompt features over T rounds. Proposed algorithms strategically select actions and observe rewards, achieving O(dT/m) regret in the full-information setting and O(dTK/m) in the bandit setting, where m is the feedback budget. Experiments demonstrate efficient learning of high-quality routing strategies across diverse LLMs from limited feedback.
- Formulates LLM prompt routing as contextual bandit with K experts, d features, T rounds
- Achieves O(dT/m) full-information regret and O(dTK/m) bandit-setting regret
- Feedback budget m (m much less than T) constrains reward observations
- Experiments show efficient routing strategies learned across diverse LLMs
From the original article
1 min read at huggingface.co
We study adaptive routing of prompts to large language model (LLM) experts to maximize response quality in an online setting with limited feedback. We formulate it as a bandit problem with K actions that represent experts and d features that encode prompts, over a horizon of T rounds. We propose algorithms that strategically select and observe rewards to minimize regret.
By Wang Wei, Soumyabrata Pal, Koyel Mukherjee · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.