Can Language Models Learn When to Abstain? Policy Learning and Calibrated Answer Selection
A learned answer selector on Qwen3-14B improves TriviaQA coverage and lowers scored-answer errors versus confidence filtering.
The paper studies whether Qwen3-14B should be trained to abstain on factual questions or whether a separate selector should check its answers, using a score that rewards correct answers, penalizes mistakes, and allows abstention. Across three training seeds and two held-out thresholds, the learned selector improved threshold-averaged utility over the base model's calibrated confidence filter. On TriviaQA, mean coverage rose from 20.2% to 30.9%, while the pooled exact-match error among scored answers fell from 23.0% to 14.5%. Tested reinforcement-learning changes did not show consistent gains, and on NQ-Open the selector still performed worse than always abstaining.
- Learned selector beat a calibrated confidence filter
- TriviaQA coverage rose from 20.2% to 30.9%
- Scored-answer error fell from 23.0% to 14.5%
- Reinforcement learning showed no consistent gains
- On NQ-Open, always abstaining still performed better
From the original article
1 min read at arxiv.orgA useful language model should know when to answer and when to say "I don't know." Can we improve that decision by training the model itself or by training a separate component to check its answers? We study both approaches on Qwen3-14B for factual questions, using a score that rewards correct answers, penalizes mistakes, and allows abstention.
By Juan Terven · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.