MultiAgent-OPD jointly trains a hierarchical agent team end to end
Original title: MultiAgent-OPD: End-to-End Training of Hierarchical Multi-Agent Systems with On-Policy Distillation
Across seven datasets the paper reports 38.68% macro avg@4, above Prompted Critic and C-MAS + RL.
MultiAgent-OPD jointly trains a shared-parameter Leader, Searcher, and Answerer with on-policy distillation and reinforcement learning. A frozen Teacher, first trained on answer correctness, scores student search and summary tokens for local process supervision. The Leader is trained on team success and the Answerer on candidate correctness. The authors say the team beats Prompted Critic by 4.06 points and C-MAS + RL by 2.61 points and solves more questions within four attempts.
- A frozen Teacher supplies process scores for Searcher tokens.
- The Leader learns from team success, not only the final answer.
- Reported macro avg@4 is 38.68% across seven datasets.
- Gains versus Prompted Critic and C-MAS + RL are 4.06 and 2.61 points.
From the original article
1 min read at huggingface.co
In a hierarchical multi-agent system, a Leader assigns tasks and subagents carry them out. Jointly training the Leader and the subagents is difficult because a final answer gives only indirect feedback to the subagents. An LLM judge can score the performance of the subagents, but these scores depend on the judge's own assessment of the quality of subagents' responses.
By Yuhan Sun, Jiaxin Zhao, Yongdong Luo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.