[AINews] 10% worse, 100x cheaper, 10000x faster: Why Simulation is taking over
Latent Space argues AI training pipeline stages—rewards, data, teachers, curricula, environments—are flipping from human-made to model-made simulation.
Latent Space's AINews essay traces how each component of AI training has turned synthetic since 2022: reward models (InstructGPT, RLAIF), synthetic pretraining data (Microsoft Phi, NVIDIA Nemotron-4 340B), model teachers (Alpaca, DeepSeek-R1 distillation), and self-generated curricula (Self-Rewarding Language Models, SPIN). In 2026 it highlights Karpathy's autoresearch loop—700 experiments yielding 20 kept improvements, cutting GPT-2 training time from 2.02 to 1.80 hours—and Z.ai's GLM-5.3 fully synthetic RL environment, judging, and verification stack. It frames these shifts as 'simulation': 10% worse but 100x cheaper and 10,000x faster than human equivalents.