#web-crawlers

0 stories this week · 1 this month · 3 in 6 months · mostly security

Stories per day · 30 days

3 stories, newest first · one card per story

About 10,000 one-off requests in seven minutes on July 6 bypassed address-based rate limits and broke search.

Cloudflare's founders say automated traffic already exceeds human web traffic and plan to pay sites agents use.

git.kernel.org spends more CPU rendering commit pages for abusive AI-era scrapers than on all legitimate access including git clones.

Stories tagged #web-crawlers