OpenAI, Google, Microsoft and Anthropic back shared AI scrape opt-out
Anthropic already honors the robots.txt token; Google and Microsoft plan support next year, and OpenAI says coming months.
58
Organization
Internet Engineering Task Force
- Mentions · 7d2
- Mentions · 30d2
- All mentions2
- First seenOct 30, 2026
- Last mentioned2h ago
Timeline
2 articles · newest firstOpenAI will honor a shared robots.txt AI training preference signal
Site operators can state one robots.txt preference for multiple providers; OpenAI says implementation is still months away.
42
Appears with
Entities are extracted by the model from each article. First seen , last mentioned . Watching keeps it in this browser only (no account); the watchlist page and dashboard alerts use it.