Thinking Outside the Box: Can Language Models Rely on External Guidance Selectively?
Box^2-Bench tests whether language models can use reliable workflows while overriding misleading external guidance.
Agent harnesses can help language models, but unreliable human-designed workflows increasingly constrain stronger models. The authors define selective reliance as benefiting from useful guidance while overriding bad guidance, and introduce Box^2-Bench, which fixes the model and task while varying workflow reliability. Frontier models often improve with reliable guidance yet remain vulnerable when it is misleading or later becomes unreliable. Training two open-weight models on bad workflows showed that counterfactual supervised fine-tuning improved robustness, while outcome-based reinforcement learning increased use of helpful workflows; the effect also aided peer correction and robustness to corrupted memory.
- Box^2-Bench varies workflow reliability while holding the model and task fixed.
- Frontier models gain from reliable guidance but still follow misleading workflows.
- Counterfactual supervised fine-tuning on bad workflows improved robustness.
- Outcome-based reinforcement learning increased use of helpful workflows.
- The behavior also improved peer correction and corrupted-memory robustness.
From the original article
1 min read at huggingface.co
Agent harnesses often improve language models with human-designed workflows, but as models grow more capable, unreliable guidance can increasingly constrain their execution. We call the ability to benefit from useful guidance while overriding unreliable guidance thinking outside the box.
By Minghan Wang, Boyuan Wang, Jinhang Zuo · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.