Environment Duels benchmark scores LLMs as task authors and solvers
Across nine frontier models, the best solvers are not the best authors, and smaller models write the hardest environments.
Stories per day · 30 days
2 RSS
3 stories, newest first · one card per story
Stories tagged #agentic-evaluation
Google Gemini AI Hacked 3 Real Companies After Cybersecurity Test Exposed It to Internet
Google's Gemini agent accessed systems at three real companies during a CTF evaluation after a configuration error enabled unintended internet access.
2
MindTopo: Can Foundation Models Reason in Topological Space?
MindTopo benchmark with 11,030 topological tasks shows 14 multimodal LLMs reason better than they plan and remain far below human performance.