Elorian launches Glimpse and the DecoyCount visual benchmark
Original title: Can Frontier Models See Past a Decoy? Elorian Tests Visual Reasoning
The paper puts frontier models under 50% accuracy; CEO Dai says Glimpse scored 50.48% on one run.
Elorian, out of stealth since April after a $55 million seed round, released its first visual reasoning model, Glimpse, and with Mercor announced the DecoyCount benchmark. The accompanying paper says frontier models scored below 50%, with GPT-6 Astra highest at 43.17%. Dai told Latent Space that Glimpse scored 50.48% on 1,660 items in a single run without a tool harness, and that evaluation images were withheld from training.
Unconfirmed: Glimpse’s score and the claim that evaluation data was withheld come only from CEO Andrew Dai.
- GPT-6 Astra scored 43.17%, the highest result reported in the paper.
- Gemini-3.8-flash scored 32.71% and Qwen3.8-max scored 28.31%.
- Claude-Fable-5.1 scored 25.22%; paper models were averaged over three runs.
- Glimpse’s 50.48% result is a single internal run and is not in the paper.
- DecoyCount tests seven counting-failure mechanisms across 1,660 items.
- Vendors
- Elorian
- Products
- DecoyCount
- Organizations
- Mercor
From the original article
5 min read at latent.space
A new decoy counting benchmark exposes a gap between seeing and reasoning. Elorian’s Glimpse tries to bridge it by thinking with images.
By Richard MacManus · latent.space
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.