Import AI 470: No rights for machines; automating environment generation with SPADE; and building better GPU kernels with Hawkeye
METR analysis finds AI accelerating cyber vulnerability discovery, while SPADE self-play environment generation improves Qwen3 reasoning benchmark scores at 30B scale.
Import AI 470 discusses a METR research note reporting differential acceleration from AI: major acceleration in reported cyber vulnerabilities (cURL, OpenSSL, Firefox, Microsoft, NVD, OSV), minor acceleration in mathematics, and no measurable acceleration in AI-research optimization benchmarks. It also covers SPADE, a self-play framework from a multi-university team (University of Washington, Stanford, MIT, CMU, and others) that co-evolves executable training environments and agent capability using Environment Designer and Reasoning Agent roles with hint-based regret rewards. Trained on Qwen3-4B-Instruct-2507, Qwen3-8B, and Qwen3-30B-A3B-Instruct-2507 via GRPO (400 rollouts of 25 environments), SPADE lifted the 30B-A3B game-environment suite average to 58.3, +8.1 over base, and improved tool-use results across backbones. The issue also references Hawkeye for building better GPU kernels.
- METR finds major AI acceleration in cyber vulnerability reporting, minor in math, none in AI research.
- SPADE co-evolves environment synthesis and agentic capability via self-play with designer and reasoning roles.
- Hint-based regret reward measures the return gap with versus without privileged hints.
- Qwen3-30B-A3B reaches 58.3 on game environments: +8.1 over base and +5.3 over fixed-environment baseline.
- GRPO tuning used 400 rollouts of 25 environments, evaluated on AIME, GPQA, LCB, Reasoning Gym.
From the original article
14 min read at importai.substack.com
Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.
By Jack Clark · importai.substack.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.