OpenAI reports coding gains on COBOL, Fortran and Visual Basic
Original title: Advancing Coding in Low-Resource Programming Languages
On OpenAI’s tests, GPT-6 Astra averaged 87.4% on MultiLangBench, up from 26.8% for GPT-5.6 Sol.
OpenAI says newer models are better at building and migrating software in low-resource languages, including COBOL, Fortran and Visual Basic. MultiLangBench asks a model to build a SQLite-compatible database engine across 20 languages. A separate evaluation migrates COBOL programs to C++, Java, Kotlin, Python or Rust with byte-for-byte output parity. OpenAI is releasing the MultiLangBench setup and results on GitHub and says production migrations still need expert review.
Unconfirmed: Pass rates and costs are OpenAI’s own evaluation figures, not an independent benchmark.
- GPT-6 Astra scored 72.1% on COBOL, 83.5% on Fortran and 89.4% on Visual Basic in that bench.
- GPT-6 Luna averaged 40.3% at about 78% lower estimated API cost than GPT-5.6 Sol.
- GPT-6.1 Sol reached 86.6% on COBOL migration at low effort versus 5.4% for GPT-5.6 Sol.
- At higher reasoning effort, GPT-6.1 Sol reached 97.9% on the migration evaluation.
- COBOL remained the hardest MultiLangBench language for Astra, at 72%.
- Vendors
- OpenAI
- Products
- MultiLangBench
From the original article
4 min read at openai.com
Learn how OpenAI is improving ChatGPT across low-resource programming languages, with new evaluations for software engineering, SQLite, and COBOL migration.
openai.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.