Performance of Clinical AI System and Physicians and Frontier Language Models in primary care diagnostics
Clinical AI system Doctorina achieved 82.0% primary-care diagnostic concordance versus 57.0% for physicians across 150 synthetic consultations.
The study compared Doctorina, eight physicians, and four standalone frontier language models on 150 synthetic Polish-language primary-care consultations. Doctorina achieved 82.0% Top-1 diagnostic concordance versus 57.0% for physicians (25.0-point difference, 95% CI 17.7-32.7) and 97.3% versus 85.0% primary-or-reference-differential concordance. Normalized workup and treatment scores were 89.4 versus 66.9 and 83.7 versus 61.2. Kimi K3 ranked next on diagnosis, while Claude Opus 5 led the closely spaced management estimates among Opus, Doctorina and Kimi.
- Doctorina: 82.0% Top-1 diagnostic concordance vs 57.0% for physicians
- Workup and treatment scores 89.4 vs 66.9 and 83.7 vs 61.2
- Kimi K3 ranked second on diagnosis among the groups
- A second Doctorina run reproduced the advantage across all outcomes
- Products
- Doctorina
- AI models
- Kimi K3Claude Opus 5
From the original article
1 min read at arxiv.orgClinical AI evaluation should encompass diagnosis and management after adaptive information gathering. We compared Doctorina, eight physicians and four standalone frontier language models in 150 synthetic Polish-language primary-care consultations. Doctorina achieved 82.
By Andy Nkansah · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.