DEPTH method synthesizes deeper terminal tasks for agents
Original title: WebHarbor: Docking Real Websites for Evolving GUI Agent Environments
DEPTH tasks exceed existing datasets in depth and work.
The authors find turn count does not reflect task depth in terminal tasks. They propose DEPTH where a generation agent writes each task segment in a real environment. Their 105 tasks exceed every audited dataset in depth and work. Fine-tuning Qwen3-8B on runs from these tasks improves performance on two terminal benchmarks.
- 94% of solved synthetic tasks pass within three turns
- 105 new terminal tasks exceed all audited datasets
- None of six models pass half of them
- Depth predicts difficulty beyond work
- Code and data released with paper
- AI models
- Qwen3-8B
From the original article
1 min read at huggingface.co
GUI web agents require realistic and interactive environments that remain stable during both evaluation and training. Live website evaluation suffers from changing content, access restrictions, and limited coverage of authenticated workflows, while existing synthetic environments for training often lack functional depth and visual fidelity.
By Zhaoyang Wang, Qianhui Wu, Boyu Gou · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.