AdviSD trains small advisors to steer frozen frontier LLMs

AdviSD uses selective self-distillation so a Qwen3-8B advisor can steer frozen Gemini and Claude, beating advisor-GRPO on two benchmarks.

Advisor Self-Distillation (AdviSD) trains a small advisor to steer a frozen language-model executor using natural-language advice. It combines outcome-based reinforcement learning with selective self-distillation: reflection proposes corrections, and only advice that meaningfully changes the score of the same recorded response is used for supervision, without executor likelihoods or additional rollouts. Qwen3-8B advisors for Gemini and Claude beat advisor-GRPO by 4.2 to 6.4 percentage points on BFCL-v3 and by 3.9 to 5.1 score points on EnvScaler in the arXiv report; the Hugging Face summary gives the same ranges as points without that unit distinction. Both sources say the advisors generalize out of domain and transfer across executor versions and model families. The reports agree on the method and the numeric ranges and disagree only on whether BFCL-v3 gains are percentage points and EnvScaler gains are score points.

• AdviSD (Advisor Self-Distillation) trains a Qwen3-8B advisor to steer frozen Gemini and Claude executors with natural-language advice.
• It pairs outcome-based reinforcement learning with selective self-distillation and does not require executor likelihoods or extra executor rollouts.
• Reflection proposes corrections; supervision uses only advice that creates a large score gap on the same recorded response.
• Versus advisor-GRPO, Qwen3-8B advisors gain 4.2–6.4 percentage points on BFCL-v3 and 3.9–5.1 score points on EnvScaler (arXiv); the Hugging Face note lists the same ranges as undifferentiated points.
• Advisors generalize out of domain and transfer across executor versions and model families.
• Sources: Hugging Face daily papers, 2026-09-28, and arXiv cs.AI/cs.LG/cs.CL, 2026-09-29; paper title AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation.