Skip to content
NewA calmer, faster ZeroHour

Logged bandit feedback trains calibrated decision heads on Qwen3

Original title: Learning a calibrated decision head from logged bandit feedback

arXiv cs.AI / cs.LG / cs.CLby Ye Yuan ()1 min read1 · 1 read
AI summary · x-ai/grok-4.7Summaries by Vani AI

CAPS beats confirmed-answer fine-tuning by 2.6 accuracy points on Banking77 and cuts ECE by 0.053.

The paper adapts a one-pass decision head with LoRA on Qwen3-0.6B-Base and 1.7B-Base using only logged bandit feedback. It finds Brier or log scores of the sampled action are unusable REINFORCE rewards and can collapse accuracy, while pathwise scoring and rewards r and r minus p avoid that. Chosen-Action Proper Scoring (CAPS) beats fine-tuning on confirmed answers only, and at 5,000 samples stays close to full-label training. Code, adapters, logs and per-record results are public.

Unconfirmed: Several reported gains, including emotion data and noisy-label rates, use one seed.

Key facts
  • One run cut ordered-scale accuracy from 0.636 to 0.111 when the reward collapsed.
  • On one emotion-data seed with an often-wrong logger, CAPS gained 21.7 points.
  • At N 5000, CAPS trailed full labels by 1.5 points at 0.6B and 0.6 at 1.7B.
  • Propensity weights added nothing on this log; early stopping blocked noisy memorization in three runs.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

A decision head reads a state and a list of options and returns a probability per option in one forward pass. Once deployed, it usually observes only whether the option it chose was right: logged bandit feedback. We adapt such a head, LoRA on Qwen3-0.

arXiv cs.AI / cs.LG / cs.CL

By Ye Yuan · arxiv.org

Read the full article at arXiv cs.AI / cs.LG / cs.CL

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.