Continuous accountability for Verified bots
Cloudflare will score Verified bots on whether they honor robots.txt and rate-limit signals over time.
Cloudflare introduced continuous scoring of Verified bots after identity is established with Web Bot Auth and BotBase. Trust bands measure whether crawlers honor robots.txt preferences, consistent with RFC 9309, and whether they slow down after HTTP 429 responses. Cloudflare cited delisting Perplexity's bot in 2025 after it ignored site opt-outs and tried to evade blocks. The system is in internal use, with public Radar scores and customer-configurable thresholds planned alongside Bot Preference Sync and Bot Management.
- Verified bots must identify themselves, do only what they claim, and avoid abuse.
- Trust bands track robots.txt compliance and responses to HTTP 429 rate limits.
- Perplexity's bot was delisted in 2025 for bypassing scrape opt-outs.
- Scores will later appear on Radar and in customer-set handling thresholds.
- Vendors
- Cloudflare
- Organizations
- CloudflarePerplexity
From the original article
5 min read at blog.cloudflare.com
Identity alone doesn't guarantee a bot will behave responsibly. We are introducing a system to continuously evaluate whether Verified bots honor site preferences, like robots. txt and HTTP 429s, ensuring they remain accountable over time.
By Chris Tsoukaladelis · blog.cloudflare.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.