Analysis finds a web crawler operating from Contabo IP space
Original title: Web Page Crawler Revealed - tracking down requests from Contabo GmbH
From August 4 to September 28, Contabo addresses fetched nearly all 1,048,578 generated pages at about 4 requests per second.
A site operator reviewed Apache logs for a generated binary-tree site and concluded at least one crawler used Contabo GmbH address space between August 1 and September 30, 2026. Separate request series each fetched nearly the full set of pages, and Series D comprised three overlapping sub-series. The operator says the traffic came from a cloud provider, not residential proxies, but does not name who ran the crawler.
Unconfirmed: The analysis identifies Contabo address space but not the crawler's operator.
- Initial probes on August 4 used 2,804 IP addresses for 3,365 requests.
- Series B, C, and E each made about 1.04 million requests near 4 per second.
- Series D logged 3,137,988 requests from 7,509 IP addresses.
- The generated site contained 1,048,578 distinct linked pages.
- Organizations
- Contabo GmbH
From the original article
11 min read at bruceediger.comThe story is a technical breakdown attempting to trace a series of crawlers from Contabo GmbH. Edited the title to be more clear. This feels relevant here but I struggled with tagging this.
By bruceediger.com via oceanhaiyang · bruceediger.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.