Dense retrievers favor passages naming high-scoring answer objects
Across eight models, higher token-prediction scores predict higher retrieval ranks even when the relation is wrong.
Stories per day · 30 days
10 RSS
Relatedreinforcement-learning6reasoning4self-distillation4on-policy-distillation3agents2compression2grpo2inceptionv12kv-cache2llm-agents2llm-training2long-context2mathematical-reasoning2mechanistic-interpretability2post-training2ppo2ruler2superposition2
82 stories, newest first · one card per story
Stories tagged #qwen3
MTA-OPSD lets each token pick its own teacher in self-distillation
On Qwen3 math tasks, the method beats prior on-policy self-distillation, with gains of up to 5.9 over the base model.
Calibrated distillation consolidates specialists in dynamic MoE models
On Turing-20B-A2B the method scores 0.9988 across four domains and beats Qwen3-8B on math and code.
Fine-tuned small models predict x86-64 register inputs from outputs
GRPO fine-tuning lifts Nemotron-3-Nano knowable-register accuracy from 63.3% to 95.2%, within four points of frontier scores.
ThinkBit preserves reasoning in scalar two-bit language models
On Qwen3-1.7B, a 0.52 GB recalibrated checkpoint scores 46.7 on MATH-500 versus 30.0 for EfficientQAT.
Trajectory2Gym builds decision-grounded supervision for tool-use agents
Across five Qwen3 scales it lifts the equal-weight Tau1, Tau2 and BFCL mean by 16.9 to 29.8 points.
NgramRL speeds RL rollouts using drafts from recent history
Warm-cache runs commit 3.60–7.77 tokens per forward, and a prefetcher raises Spider throughput 1.38x.
Safety mid-training lowers LLM attack success before post-training
Authors report a 72% to 81% relative attack-success drop versus Qwen3 1.7B, 4B, and 8B bases.
Prompt-injection outcomes often form after the agent reads
Thinking cuts Qwen3 variance fixed at read time from 0.91 to 0.62, and single-run labels disagree up to 35%.
Is AI interpretability close to a breakthrough? A new essay has researchers divided
Nick Cammarata argues famous polysemantic neurons were misread, claiming most Qwen3-32B neurons have one job.
7
Neural Networks Are Already Clean
A Goodfire researcher argues neural networks are already interpretable and superposition was misread.
12
Athanor: An Agent-Native System for Autonomous Post-Training Research
Athanor agents autonomously post-trained Qwen3-4B-Base to match official Qwen3-4B-Instruct on held-out tasks.
Waypoint-OPD: Transferring RL-Induced Policy Shifts via Gated On-Policy Distillation
Waypoint-OPD gates RL policy-shift distillation and leads math and code scores for Qwen3 students.
Latent Compaction with Test-Time Training
Latent Compaction preserves long-context retrieval at 32x KV-cache compression via test-time training.
Recovering Reliable Supervision via Critique-Guided On-Policy Distillation
CriticOPD uses teacher critiques to restore recoverability in on-policy distillation, lifting two Qwen3 students on math benchmarks.
VASPO: Self-Distillation with Privileged Value Estimation
VASPO combines privileged self-distillation with PPO, gaining up to 2.59 points over PPO on math benchmarks.
Hindsight-Privileged RLVR: Single-Rollout Token Credit Without Group Sampling
HP-RLVR gives single-rollout token credit for RLVR, training Qwen3 faster than GRPO with comparable accuracy.
τ-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions
τ-Emotion-Bench shows emotional users expose reliability gaps in tool-calling LLM agents.
ActionOPD: Distilling Decisions, Not Tokens
ActionOPD distills black-box teacher decisions into Qwen3 agents, gaining 3.3-6.9 points out of distribution.
VASPO: Self-Distillation with Privileged Value Estimation
VASPO blends a privileged self-teacher with PPO, improving math accuracy on Qwen3-4B and OLMo3-7B.
Latent Compaction with Test-Time Training
Latent Compaction keeps 96.7% RULER accuracy at 32x KV-cache compression via test-time training.
Unlocking Structured Pruning for Reasoning Language Models via Attention-Preserving MLP Trimming
LayerTrim prunes reasoning LLMs without training, cutting cost while preserving long-form accuracy.
LeanPRM: Lean-Grounded Process Supervision for Natural Language Proof Verification
LeanPRM uses Lean to auto-label 152K proof steps, lifting a Qwen3-4B verifier's step accuracy from 48.2 to 74.7.
ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
ANTE lifts Qwen3-32B to 57.2% pass@1 on SWE-bench Verified using verified action forecasts.