EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation
EdgeWise delegates only key agent steps to the cloud, cutting inference cost by up to 62%.
EdgeWise is an edge-cloud agent framework that runs a small language model locally and selectively calls a cloud LLM for critical steps of long-horizon tasks. A step-level reinforcement-learning method jointly optimizes solving and delegation, with credit assignment for individual delegation choices; the authors also study a judge-based strategy and design choices such as timing, context sharing, and call frequency. Experiments match or beat full cloud-LLM execution while cutting inference cost by up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1. A local workstation deployment shows comparable savings.
- A local SLM solves most steps and delegates hard ones to a cloud LLM.
- Step-level RL jointly trains task solving and delegation decisions.
- Cost falls up to 62% on SWE-Rebench and 34% on Terminal-Bench 2.1.
- A workstation deployment shows similar savings in a practical edge-cloud setup.
From the original article
1 min read at huggingface.co
Complex and long-horizon agent tasks often rely on frontier models deployed in the cloud to achieve strong performance. However, using these models for every step of a long-horizon trajectory incurs substantial inference cost, even though not all steps may require the same level of model capability.
By Zeru Shi, Jindong Jiang, Ryo Hachiuma · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.