Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows
Argo-Bench tests data agents on 210 enterprise tasks over a 7.5-billion-row simulated warehouse.
Argo-Bench evaluates data agents on 210 enterprise analytics tasks that go beyond text-to-SQL. A simulated New York City food-delivery platform, with 81 million orders in 2024, is exported to an Oracle E-Business Suite-style warehouse of 235 tables and 7.5 billion rows. Agents must reconstruct withheld facts and take actions such as banning fraudulent accounts or allocating courier incentives, scored by simulator consequences. The strongest of 14 frontier and open-weight models averages 59.5 points and reaches 95 or higher on only 34.8% of tasks.
- Argo-Bench has 210 enterprise analytics tasks graded by simulated consequences.
- Warehouse holds 235 tables and 7.5 billion rows from 81 million 2024 orders.
- Ground truth is withheld, so agents must reconstruct facts before acting.
- Best of 14 models averages 59.5 and scores 95+ on only 34.8% of tasks.
- Vendors
- Oracle
- Products
- Oracle E-Business Suite
- Countries
- United States
From the original article
1 min read at huggingface.co
Real-world enterprise data science and analytics workflows require reasoning across dozens of tables, performing statistical analyses, and acting on the results. Established text-to-SQL benchmarks evaluate query generation alone, and audits have found their answer keys frequently wrong.
By Gabriel Tomitsuka, Arman Raayatsanati, Emma Xing · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.