Trained agent policies carry harness preferences that break under interface changes
Original title: Your Policy Is Secretly a Harness: The Decision Geometry of Agent Interfaces
Reordering MiniWoB++ observation fields drops trained success from 85.0% to 19.2% without changing optimal decisions.
The paper argues that, under KL-regularized control, an optimal policy's log odds recover how strongly its harness favors each action, so a trained policy carries that harness's preferences. Deployed return equals the harness ceiling minus mismatch from suboptimal decisions; training changes only mismatch, while a new harness moves both. In MiniWoB++, reversing observation-field order leaves every optimal decision unchanged but cuts success from 85.0% to 19.2%. On Terminal-Bench Lite, ceiling minus mismatch estimated with a stronger stand-in model reproduces accuracy rankings. Code is released as HRP.
- Harnesses rejected in fixed-model comparisons can beat the original after further training if they raise the ceiling.
- Ceiling and mismatch measured beforehand predict those post-training gains.
- Either term alone does not rank coding-agent harnesses on Terminal-Bench Lite.
- The derivation assumes KL-regularized control.
- Products
- MiniWoB++Terminal-Bench Lite
From the original article
1 min read at huggingface.co
Language model agents are post-trained and deployed through a harness, the prompts, tools, and control logic that present a task to the model and execute its outputs. Harnesses are often revised and judged by how an already trained model performs under them, which assumes that what the model learned under one harness carries over to the next.
By Jian Mu, Ruibo Shen, Yao Shu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.