ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “Bot Preference Sync”

2 stories in the last 30d

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllTools 2
SeverityAllcritical 0high 0medium 0low 0info 2

Say it once: Introducing Bot Preference Sync

Cloudflare launched Bot Preference Sync, which automatically syncs robots.txt files with AI bot policies covering search, agent, and training crawlers.

Cloudflare's new Bot Preference Sync feature automatically aligns a site's robots.txt with its configured AI bot policies for Search, Agent, and Training bot categories. The goal is to let site owners manage which automated and AI crawlers access their content without manually maintaining static robots.txt files.

Cloudflare Blog · 25d agoTools

Have it both ways: stay discoverable in search while disallowing AI training

Cloudflare launches Disallow AI Training setting letting sites block AI training via robots.txt while staying indexed in search; Apple, Google, and Microsoft honor it.

Cloudflare announced a 'Disallow AI Training' setting that publishes a no-training preference in robots.txt so sites can block AI training (including by mixed-use crawlers) without losing search indexing. Apple, Google, and Microsoft meet Cloudflare's new 'Accountable' designation, which requires training/summary opt-out mechanisms, URL-level training visibility, and assurance that opt-outs don't affect search rankings. Cloudflare cites that under 1% of its sites block search bots while 17% block AI training, and its Block settings now apply to mixed-use crawlers as well. Granular controls over how much content appears in AI summaries are planned for early next year.

Cloudflare Blog · 1d agoTools