Skip to content
NewA calmer, faster ZeroHour

OpenAI publishes pretraining perplexity to anchor shared safety standards

Original title: Perplexity and Intelligence

OpenAI News ()6 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

The post cites log-perplexity of about 0.99 for GPT-3.5, 0.91 for GPT-4, and below 0.8 for newer models.

An OpenAI post argues that pretraining progress is best tracked by perplexity, or cross-entropy loss, and that the metric should compare developers and inform shared safety standards. It says internal measurements from GPT-3.5 onward show capabilities emerging as loss falls, including fluent chat near 0.99, chain-of-thought near 0.91, and cybersecurity and mathematics below 0.8. A model near 0.76 is said to have produced notable mathematics results. The author expects further changes in a roughly 0.68–0.7 window and says algorithmic pretraining advances could arrive as soon as 2027.

Unconfirmed: Capability thresholds, the 0.68–0.7 window, and a 2027 pretraining avalanche are the author's extrapolations, not demonstrated results.

Key facts
  • GPT-3.5 is given at about 0.99 log-perplexity and GPT-4 at about 0.91.
  • Losses below 0.8 are tied to cybersecurity and mathematics capabilities.
  • A model near 0.76 is said to have resolved long-standing math questions.
  • The author flags a possible step change from AI doing pretraining research.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

6 min read at openai.com

How perplexity measures progress in AI models, and why it could help developers establish a shared approach to evaluating emerging capabilities.

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.