TuringSpace tests spatial reasoning with interactive CAPTCHAs
Original title: TuringSpace: Grounding Spatial Intelligence in Action with Interactive CAPTCHAs
Humans solve 94.1 percent; GPT-6 Astra fails 25.9 percent and Kimi K3 fails 68.0 percent.
TuringSpace is a synthetic web environment of 125 interactive CAPTCHAs covering five spatial abilities, solved by clicks and drags. Each task is a program that generates instances with a checker and an oracle, so the set can also train models. The authors say most failures come from misreading screenshots and that reasoning rarely corrects a wrong belief. Fine-tuning Qwen3.5-9B raised its success rate from 1.6 percent to 33.1 percent and improved seven out-of-distribution benchmarks. The environment will be released.
Unconfirmed: Benchmark scores and failure analysis are author-reported and not independently confirmed here.
- Best open-weight model Kimi K3 fails 68.0 percent of tasks.
- Qwen3.5-9B rose from 1.6 percent to 33.1 percent success.
- That fine-tune scored above Kimi K3.
- VSI-Bench indoor videos gained 9.9 points.
- Tasks span five spatial abilities and form a training gym.
- Products
- TuringSpaceVSI-Bench
- AI models
- GPT-6 AstraKimi K3Qwen3.5-9B
From the original article
1 min read at huggingface.coHumans act in the visual world by inferring the 3D layout behind a 2D view, predicting how an action changes it, and correcting themselves after seeing the result. Can multimodal large language models (MLLMs) do the same? CAPTCHAs offer a natural test, since humans solve them through this very loop, clicking and dragging on the screen and adjusting after each result.
By Xuwei Ding, Skylar Zhai, Linxin Song · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.