Harness Learning Enables Generalizable Test-Time Adaptation
Harness learning trains a proposer to revise an agent's executable harness from execution feedback.
The paper treats a language-model agent as a model plus a harness that organizes model calls, tool use, and information flow. Harness learning trains a proposer with reinforcement learning to revise the solver's harness, using the revised harness's task performance as the reward and framing revisions as meta-learning over executable programs. At test time the proposer refines the harness from successive execution feedback without updating model parameters. Experiments on reasoning and multi-hop question answering show better revisions and transfer of test-time adaptation to unseen tasks.
- A proposer revises a solver harness using execution feedback.
- Reinforcement learning rewards revised harnesses by task performance.
- Test-time adaptation requires no parameter-space model updates.
- Adaptation transfers to unseen reasoning and multi-hop QA tasks.
From the original article
1 min read at arxiv.orgA language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Because different tasks call for different ways of organizing these operations, the harness needs to be adapted using feedback from the task at hand. We introduce harness learning, which trains a proposer model to revise a solver's harness using execution feedback.
By Alvin Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.