RATED: Reliability-Aware AI Agent Tool Retrieval via Execution Dynamics
RATED re-ranks AI agent tools from execution feedback, cutting fault selection from 86.2% to 2.9%.
RATED pairs a frozen contrastively fine-tuned dual-encoder with a lightweight contextual bandit that re-ranks tools from real-time execution feedback. The EcoAgent benchmark contains 5,172 real-world tools and more than 210,000 query-tool pairs. RATED reports a 52.2% relative MRR improvement over the ToolLLM API retriever and cuts fault selection from 86.2% to 2.9% when 30% of APIs fail, with under 1 ms overhead. Code and data are released at github.com/Elliotes/EcoAgent.
- Frozen dual-encoder plus a contextual bandit adapts without retraining
- EcoAgent covers 5,172 tools and over 210,000 query-tool pairs
- 52.2% relative MRR gain versus the ToolLLM API retriever
- Fault selection falls from 86.2% to 2.9% at a 30% failure rate
- Added latency stays under 1 millisecond
From the original article
1 min read at arxiv.orgAs AI agents increasingly rely on external tools, accurate retrieval is essential for reliable behavior. Existing methods optimize semantic relevance but largely ignore tool availability during execution. To address this gap, we propose RATED (Reliability-Aware Tool Retrieval via Execution Dynamics), a retrieval framework that combines robust semantic matching with dynamic reliability adaptation.
By Weiliang Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.