AI models struggle with low-latency tasks
Original title: Import AI 479: Spacewar; low-latency tasks versus LLMs; Genie 4
Realtime-Bench shows frontier LLMs perform poorly on real-time interactive tasks compared to paused-clock evaluations.
Researchers at University of Texas at Austin, Ohio State University and others released Realtime-Bench, a suite of 50 dynamic web and game tasks to evaluate AI agents on temporal state awareness, adaptive deliberation and time-aware action selection. Top models scored 36% on wall-clock setting but 70-80% on paused-clock. Separately, the Defense Intelligence Agency released a report on space-based directed energy, jamming and satellite-on-satellite attacks.
Unconfirmed: Exact performance figures for all tested models not detailed
- 36% pass rate for top model on wall clock
- 50 tasks in Realtime-Bench
- Models tested include GPT-6-Astra, Claude Opus 5, Gemini 3.8 Flash
- DIA report details China-Russia space conflicts
From the original article
10 min read at importai.substack.com
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.
By Jack Clark · importai.substack.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.