Cloudflare and OpenAI: making AI discovery better for everyone
A Cloudflare-OpenAI pilot helped ChatGPT index hundreds of millions of new pages from about one percent of Cloudflare's network.
Cloudflare and OpenAI say a research pilot started in July used signals from about 1% of Cloudflare’s network to help ChatGPT discover content. OpenAI independently ranked pages and indexed hundreds of millions of previously unknown URLs for ChatGPT Search. The signals also filter missing pages, blanket redirects, duplicate content, and individual harmful pages such as planted fake login forms, using metadata rather than page content for training. Cloudflare says customers can still manage crawler access with AI Crawl Control; the post also references GPT-6 and Intelligent UI.
- Pilot covered roughly 1% of Cloudflare’s network since July.
- Hundreds of millions of new pages entered OpenAI’s search index.
- Signals skip dead URLs, redirects, duplicates, and single harmful pages.
- Metadata supports discovery, not training; AI Crawl Control remains available.
- Vendors
- CloudflareOpenAI
- Organizations
- CloudflareOpenAI
- AI models
- GPT-6
From the original article
4 min read at blog.cloudflare.com
How Cloudflare and OpenAI are working together to help AI find useful content, keep up with changes, and make fewer wasted requests — making the experience better for users and website owners alike.
By Matthew Conroy · blog.cloudflare.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.