ASTOR turns natural-language soft constraints into optimization code
Original title: ASTOR: Benchmarking and Improving Autoformulation for Soft-Constraint Optimization with a Formal Grammar
On the new SoftOR benchmark, the authors report the best aggregate accuracy among compared LLM methods.
ASTOR maps a natural-language decision problem to a grammar-guided abstract syntax tree, then deterministically compiles that representation into executable optimization code. SoftOR is a benchmark centered on soft constraints, which may be violated, unlike hard requirements. The authors say ASTOR leads comparable single-request and multi-agent methods on SoftOR and remains competitive on hard-constraint benchmarks. Code and data are posted at an anonymous repository.
Unconfirmed: Accuracy rankings are the authors' own benchmark results.
- Compilation from the grammar-guided syntax tree into optimization code is deterministic.
- SoftOR focuses on preferences that can be violated rather than only hard constraints.
- Code and data are available through an anonymous 4open.science repository.
From the original article
1 min read at arxiv.orgReal-world decision problems combine two qualitatively different kinds of constraints: requirements that must be satisfied, called hard constraints, and preferences that can be violated, called soft constraints. Because these problems are often posed to decision-makers in natural language, large language models offer a natural tool for assisting with their solution.
By Amer Goel · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.