Trained agent policies carry harness preferences that break under interface changes
Reordering MiniWoB++ observation fields drops trained success from 85.0% to 19.2% without changing optimal decisions.
Stories per day · 30 days
Relateddistillation2small-language-models2agent-harness1agent-training1fine-tuning1harness1human-ai-collaboration1maze-navigation1miniwob1on-policy-distillation1post-training1shared-control1terminal-bench1
4 stories, newest first · one card per story
Stories tagged #language-model-agents
Stronger models defer maze turns only when humans hold the advantage
Reliable solo solvers almost never deferred without a human route advantage, but deferred on about 80% of initial decisions when the human path was shorter.
Progress-Aware Distillation targets stagnation in small agent models
Tests on eight models from 0.6B to 8B parameters show higher success rates and faster task completion.
Harness-Aware Distillation for Small Language Model Agents
Harness-Aware Distillation trains smaller agents on skills the fixed software harness cannot supply.