Paper shows adaptive AI solvers recover after CAPTCHA changes
Original title: Training AIs to Solve CAPTCHAs
A YOLOv8 detector routed to an open-weight vision-language model reached 85.4% accuracy and retrains from its own labels.
A research system pairs a fine-tuned YOLOv8 detector with an open-weight vision-language model and a confidence-based router. It works from screenshots and operating-system input only, with no browser automation or DOM access. Answers from the vision-language model become training labels, so the detector learns new CAPTCHA classes after one or two encounters. In a simulated monthly arms race, the solver recovered after each perturbation round.
Unconfirmed: The year-long arms race is simulated, not a report of real attackers using the system.
- Overall accuracy was 85.4%, with 84.2% macro accuracy across 16 classes.
- Perturbations drove the public detector to 0% while leaving the vision-language model intact.
- A roughly 70%-accurate open-weight teacher hardened the detector as well as a perfect oracle.
- The system needs no human annotation for newly absorbed categories.
From the original article
1 min read at schneier.comInteresting research: Abstract: Vision-language models (VLMs) can solve visual CAPTCHAs without task-specific training, but the agents built on them approach every challenge from scratch. For such an agent, the hundredth instance of a familiar puzzle costs as much time and compute as the first. Specialized detectors invert the trade-off, answering in milliseconds but only for categories they were trained on.
By Bruce Schneier · schneier.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.