[AINews] 10% worse, 100x cheaper, 10000x faster: Why Simulation is taking over
Latent Space argues AI training pipeline stages—rewards, data, teachers, curricula, environments—are flipping from human-made to model-made simulation.
Latent Space's AINews essay traces how each component of AI training has turned synthetic since 2022: reward models (InstructGPT, RLAIF), synthetic pretraining data (Microsoft Phi, NVIDIA Nemotron-4 340B), model teachers (Alpaca, DeepSeek-R1 distillation), and self-generated curricula (Self-Rewarding Language Models, SPIN). In 2026 it highlights Karpathy's autoresearch loop—700 experiments yielding 20 kept improvements, cutting GPT-2 training time from 2.02 to 1.80 hours—and Z.ai's GLM-5.3 fully synthetic RL environment, judging, and verification stack. It frames these shifts as 'simulation': 10% worse but 100x cheaper and 10,000x faster than human equivalents.
- Reward, critique, and evaluation ran on models judging models since 2022
- Synthetic textbook-quality data (Phi, WRAP, Nemotron-4 340B) industrialized pretraining corpora
- Karpathy's autoresearch stacked 700 experiments into 20 kept overnight improvements
- Z.ai GLM-5.3 ships end-to-end synthetic RL environments, judges, and verifiers
- AI models
- AlphaEvolveAI Scientist
From the original article
13 min read at latent.space
By AI standards today is a pretty quiet Friday, so it’s time to take a step back and reflect on what is really going on. If you read our 2025 reading list , and followed our coverage of Z.
latent.space
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.