JFinBench scores 12 LLMs on 2026 Japanese finance exams
Original title: JFinBench: A Continuously Refreshable Japanese Financial Benchmark from the Latest Certification Exams
Claude Fable 5 and GPT-6 Astra exceed 93 percent; Kimi K3.1 Pro trails the leader by 12 points.
Researchers released JFinBench, 321 examples from the 2026 editions of six Japanese certification exams in accounting, tax, financial planning, and banking. A human-in-the-loop pipeline converts heterogeneous exam formats so the set can be refreshed and limit contamination from leaked questions. With a terminal and web access, every tested model improves, but about a 4-point gap remains and rankings stay largely unchanged. The dataset, evaluation code, and pipeline are released.
- The set draws on six regularly administered certification exams.
- Claude Fable 5 and GPT-6 Astra exceed 93% overall.
- Best open-weight model Kimi K3.1 Pro trails by 12 points.
- Tool access narrows the remaining gap to about 4 points.
- Periodic refreshes are meant to limit training-data contamination.
- Countries
- Japan
From the original article
1 min read at huggingface.coLarge language models (LLMs) are increasingly applied to finance, creating a need for benchmarks that evaluate their capabilities across financial systems with different regulations and practices. However, existing financial benchmarks provide limited coverage of country-specific financial systems, and they are difficult to keep up to date as released questions leak into training data.
By Issa Sugiura, Taro Makino, Takashi Ishida · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.