MASCAL trains multi-agent systems to pass calibrated confidence
Original title: Learning Calibrated Confidence for Multi-Agent Collaboration
Upstream agents hand off a correctness probability so downstream agents can trust, verify, or recheck web and reasoning answers.
MASCAL trains each agent to emit a correctness probability by jointly optimizing task performance and confidence calibration with reinforcement learning. The upstream probability is passed at handoff, and the final answer also carries a calibrated probability. Confidence-aware weighting increases learning on positions that respond to the upstream signal. Across three collaboration architectures for web retrieval and reasoning, the method improved accuracy and calibration and generalized to deeper chains and different model scales.
- Intervention analysis found upstream confidence changes downstream decisions.
- Learned final confidence supports selective answering and targeted rechecking.
- Results cover web retrieval and multi-hop reasoning collaboration.
- The method generalized to deeper chains and different model scales.
From the original article
1 min read at arxiv.orgComplex information seeking on the Web often requires collecting evidence from multiple documents and reasoning across them, as in multi-hop question answering. Multi-agent systems address such tasks by dividing work among specialized agents, where upstream agents produce intermediate answers and supporting evidence that downstream agents build upon.
By Chenxiao Fan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.