False Frontiers: Diagnosing and Mitigating Co-Cheating in Self-Evolving Search Agents
CrossFit cuts co-cheating in self-evolving search agents and beats coupled self-evolution on seven benchmarks.
Self-evolving search agents can co-cheat: a proposer and solver agree on shared errors, so internal reward rises while external correctness stalls. Multi-sample verification only modestly reduces false-agreement mass and costs six extra generations per candidate. CrossFit splits source documents so questions from one group are scored by a solver trained only on the other, blocking same-source pseudo-label feedback. On Qwen3.5-4B and Qwen3.5-9B, CrossFit cuts false-agreement to 3.0% and 3.7% and improves average scores on seven search benchmarks by 8.8 and 8.4 points over coupled self-evolution.
- Co-cheating grows across self-evolution rounds as pseudo-labels stagnate.
- MSV trims false agreement only slightly, at six extra generations.
- CrossFit lowers false-agreement to 3.0% (4B) and 3.7% (9B).
- Gains of 8.8 and 8.4 points over coupled self-evolution.
Full article275 words · extracted from huggingface.co · click to collapse
Self-evolving search agents build their own training curricula by jointly optimizing a proposer that generates questions and a solver that answers them. This closed loop introduces a failure mode we call co-cheating: the proposer and solver increasingly agree on shared errors, so internal reward improves without a matching gain in external correctness. A post-hoc audit against source evidence shows co-cheating growing more severe over successive rounds of self-evolution, with pseudo-label correctness stagnating or declining even as the in-loop training signal improves. The most direct mitigation is to verify proposals before training: we introduce multi-sample verification (MSV), which queries the same model three times with the source and three times without it to decide task admission and replace unreliable pseudo-labels. MSV partially reduces false agreement but leaves substantial residual co-cheating and costs six extra labeler generations per candidate. These limitations motivate CrossFit, our main method: it partitions the proposer's source documents into groups A and B; questions generated from A are scored by an auxiliary solver trained only on B, and vice versa. The cross-fitted agreement determines proposer reward, so a same-source pseudo-label cannot be reproduced through the feedback solver, while the original solver's update rule is unchanged. Rerunning the loop with Qwen3.5-4B and Qwen3.5-9B, MSV reduces false-agreement mass from 6.1% to 5.7% and from 8.8% to 7.2%, whereas CrossFit reduces it to 3.0% and 3.7%. Replaying identical proposals with source-excluded feedback further reduces false agreement to 0.4% and 0.1%, isolating feedback ancestry from curriculum changes. Across seven downstream search benchmarks, CrossFit improves average performance over standard coupled self-evolution by 8.8 and 8.4 points and over Search-R1 by 8.7 and 7.8 points at 4B and 9B.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.39102