Say it once: Introducing Bot Preference Sync
Cloudflare launched Bot Preference Sync, which automatically syncs robots.txt files with AI bot policies covering search, agent, and training crawlers.
Cloudflare's new Bot Preference Sync feature automatically aligns a site's robots.txt with its configured AI bot policies for Search, Agent, and Training bot categories. The goal is to let site owners manage which automated and AI crawlers access their content without manually maintaining static robots.txt files.
- Automatic synchronization of robots.txt with Cloudflare bot policies
- Covers Search, Agent, and Training AI bot categories
- Removes need to hand-maintain static robots.txt files
- Vendors
- Cloudflare
- Products
- Bot Preference Sync
From the original article
Cloudflare's new Bot Preference Sync automatically aligns your robots. txt file with your AI bot policies for Search, Agent, and Training. Easily manage which bots access your content without maintaining static files.
By Jin-Hee Lee · blog.cloudflare.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.