AuXSpec trains Lean specification models with executable rewards
Original title: AuXSpec: Learning to Autoformalize Program Specifications via Executable Postconditions
Authors release trained Qwen3.5-9B and Qwen3.8-27B weights; the 27B model led SpecGen pass@4 at 84.3%.
AuXSpec is a Lean framework that pairs declarative postconditions with executable counterparts so test cases can score specifications by execution and supply a reinforcement-learning reward. The authors synthesized 3,037 LeetCode problems and expanded training to 19.5K samples via expert iteration on Codeforces. They report that training Qwen3.5-9B and Qwen3.8-27B raised SpecGen pass@4 on AuXSpec-Test from 0.7% to 82.7% and 14.3% to 84.3%, with the 27B model outperforming frontier models. On Verina, both models reached 90.5% and beat VeriSpecGen-30B-A3B. Weights, 3,037 problems, and the 300-problem AuXSpec-Test set are released.
Unconfirmed: Benchmark gains and the claim of beating frontier models are the authors' reported results.
- Verina pass@1 rose from 7.0% to 90.5% and 21.9% to 90.5%.
- Prior open-source leader VeriSpecGen-30B-A3B scored 47.6% pass@1 and 63.9% at pass@10.
- AuXSpec-Test includes expert review and correctness proofs.
- Models and data are on the eth-algopt Hugging Face collection.
From the original article
1 min read at huggingface.coFormal specifications define intended program behavior and underpin verification, but writing them in theorem provers such as Lean is slow and expertise-intensive, making specification a major bottleneck to scaling formally correct code.
By Haocheng Wang, Weixuan Yuan, Florian Hübler · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.