Weak-acting decision models still improve long-horizon agent policies
Original title: Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents
JEV as a teacher raises a Qwen2.5-1.5B policy from 72.8% to 82.9% on ALFWorld and from 56.8% to 71.4% on WebShop.
The paper compares three decision models and three general LLMs as actors and judges on ALFWorld, then uses JEV as a teacher on ALFWorld and WebShop. As actors, all six models favor an easier subgoal over one that must come first. As judges of a full trajectory, they credit progress toward the required subgoal, and asking whether each action helped separates useful from useless steps. The authors argue that judging after the episode reveals consequences acting must anticipate. Code is public.
Unconfirmed: Success rates are figures reported by the paper's authors.
- Six models are tested as actors and judges on ALFWorld.
- Judges are asked whether each action helped rather than whether the task succeeds.
- The student policy is Qwen2.5-1.5B trained over GRPO.
- Code is at github.com/ZJU-REAL/decision-agents.
- Organizations
- ZJU-REAL
- AI models
- JEVQwen2.5-1.5B
From the original article
1 min read at huggingface.coDecision models answer a question with a typed output, such as a choice among options with a probability for each, so one model can choose an agent's actions, evaluate actions already taken, or score every action to teach a policy with reinforcement learning. Prior work has studied them mainly as actors, judging their usefulness by how well they act.
By Haochen Liu, Qianhao Zhu, Zixuan Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.