Fine-tuned Inception v3 beats GPT-6 Astra and Claude on skin cancer
Original title: A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis
On 200 biopsy-proven MILK10k lesions, the CNN named the diagnosis for 125 cases versus 84 and 85 for the LLMs.
A dermatologist compared GPT-6 Astra and Claude Fable 5.1 with a fine-tuned Inception v3 on 200 histopathology-confirmed lesions from the public MILK10k set. Each case had a clinical photo and a dermoscopic image, and models chose among six diagnoses and whether the lesion could be discharged. The CNN, trained on ISIC images with the test lesions held out, was correct on 62% of cases; the two LLMs were each correct on 42%.
Unconfirmed: Results come from one dermatologist's test and are not described as a peer-reviewed multi-site trial.
- The set had 100 lesions not safe to discharge and 100 that were.
- Melanoma accuracy was 8 of 30, 11 of 30 and 13 of 30.
- GPT-6 Astra named all 25 basal cell carcinomas; Claude named 18.
- Inception v3 was fine-tuned on 68,011 images and can run in a phone browser.
- Neither LLM refused a case; both received the same prompt at default settings.
- Vendors
- Organizations
- International Skin Imaging Collaboration
From the original article
7 min read at lynchlab.netI am a dermatologist. I tested the ability of GPT-6 Astra and Claude Fable 5. 1 to diagnosis skin cancer and found that they were less accurate than I expected.
By sungam · lynchlab.net
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.