Skip to content
NewA calmer, faster ZeroHour

JFinBench scores 12 LLMs on 2026 Japanese finance exams

Original title: JFinBench: A Continuously Refreshable Japanese Financial Benchmark from the Latest Certification Exams

Hugging Face daily papersby Issa Sugiura, Taro Makino, Takashi Ishida ()1 min read1
AI summary · x-ai/grok-4.7Summaries by Vani AI

Claude Fable 5 and GPT-6 Astra exceed 93 percent; Kimi K3.1 Pro trails the leader by 12 points.

Researchers released JFinBench, 321 examples from the 2026 editions of six Japanese certification exams in accounting, tax, financial planning, and banking. A human-in-the-loop pipeline converts heterogeneous exam formats so the set can be refreshed and limit contamination from leaked questions. With a terminal and web access, every tested model improves, but about a 4-point gap remains and rankings stay largely unchanged. The dataset, evaluation code, and pipeline are released.

Key facts
  • The set draws on six regularly administered certification exams.
  • Claude Fable 5 and GPT-6 Astra exceed 93% overall.
  • Best open-weight model Kimi K3.1 Pro trails by 12 points.
  • Tool access narrows the remaining gap to about 4 points.
  • Periodic refreshes are meant to limit training-data contamination.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

Large language models (LLMs) are increasingly applied to finance, creating a need for benchmarks that evaluate their capabilities across financial systems with different regulations and practices. However, existing financial benchmarks provide limited coverage of country-specific financial systems, and they are difficult to keep up to date as released questions leak into training data.

Hugging Face daily papers

By Issa Sugiura, Taro Makino, Takashi Ishida · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.