Autonomous Research for Open-Ended Problems: A Case Study on Telecom Ticket Retrieval
Case study shows autonomous LLM research reaches 90% of SOTA on telecom ticket retrieval in 10 weeks versus 10 months human work.
The paper explores adapting autonomous research to open-ended, industry-grade ML problems through a telecom ticket retrieval case study with commercial and open-source agents. Autonomous research reached 90% of state-of-the-art performance (0.34 vs. 0.38 Recall@1) in 10 weeks versus 10 months of human work, at up to $200 per Cursor campaign. The authors find agents excel at narrow hyperparameter optimization but lack human-like intuition, recommending human-agent collaboration.
- Autonomous research achieved 0.34 vs. 0.38 Recall@1 SOTA on telecom ticket retrieval
- 10 weeks of agent work versus 10 months of human effort, up to $200 per campaign
- Agents excel at hyperparameter optimization but lack human-like intuition and creativity
From the original article
1 min read at arxiv.orgRecent breakthroughs in LLM-based systems and their abilities in problem solving and coding have allowed progress in the AI for Science paradigm, potentially replacing human roles in machine learning (ML) research.
By Junghyun Min · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.