LoRA dynamics differ between supervised fine-tuning and reinforcement learning
RL concentrates LoRA spectrum while SFT expands it at moderate rates, with group-mean baseline subtraction identified as the source.
Stories per day · 30 days
1 RSS
Relatedreinforcement-learning5continual-learning2foundation-models2pretraining2qwen2agent-evaluation1agentic-search1agents1agent-training1audio-language-models1benchmarks1coworkenv1deep-research1lora1multi-agent1multi-domain1music1orchestration1
13 stories, newest first · one card per story
Stories tagged #supervised-fine-tuning
Paper unifies foundation-model pretraining, SFT and RL under one algorithm
A single likelihood trainer mixes static data and RL generations and is reported to improve the accuracy-compute tradeoff.
Paper unifies pretraining, SFT, and RL under one likelihood algorithm
The authors recast RL stages as maximum-likelihood fits to model-generated targets and mix them in one trainer.
ParetoRSI-Gym evaluates multi-domain recursive self-improvement
Claude Code with Opus-5 raised equal-weighted generalization scores 9.2% relative, or 2.3 absolute points, over the starting models.
DynaFlow lets a main agent synthesize multi-agent workflows
With a fixed Qwen3.5-27B backbone, Forced mode reached 88.08% overall, 10.90 points above the strongest baseline.
Trajectory2Gym builds decision-grounded supervision for tool-use agents
Across five Qwen3 scales it lifts the equal-weight Tau1, Tau2 and BFCL mean by 16.9 to 29.8 points.
PatEx-Gym benchmarks agents on patent prior-art search
A 4B PatEx-Agent trained on synthetic trajectories is presented as competitive with much larger general-purpose agents.
MusicTwins post-trains Audio Flamingo Next to match a listener persona
On ten held-out personas, fine-tuning and reinforcement learning improved personality, style, and music responses.
CoworkEnv synthesizes training environments for long-horizon coworker agents
SFT on Qwen3-VL-8B-Instruct raised the Claw-Eval mean from 46.6 to 64.1, including a 31.9-point multi-turn gain.
Architect-Ant: Editable Automatic Furnishing of Architectural Floor Plans
Architect-Ant learns professional furnishing from 505 real floor plans and generates editable furniture layouts.
On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training
Paper shows constraining SFT parameter updates to on-policy update directions (OPSFT) transfers on-policy generalization gains to supervised fine-tuning.
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
ActObs supervises observation tokens during SFT, improving GRPO-trained agent pass@k on Terminal-Bench 2.0 and cross-domain code editing for Qwen3 models.
Everything in Moderation: Per-Domain Coverage Optima and Alignment-Resistant Domain Gaps in Multi-Domain Mid-Training
Controlled mid-training experiments on Qwen3-8B-Base find each domain has a 10-40% coverage optimum and domain gaps survive alignment SFT.