ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “web crawling”

9 stories in the last 30d

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllTools 4Research 3AI industry 1Industry 1Phishing & fraud 1
SeverityAllcritical 0high 0medium 2low 0info 8

You Get What You Sample: Evaluating Sampling Strategies for Web Security Measurements

Evaluation of 500k Tranco and 24.8M Common Crawl hosts shows Top-N domain sampling biases web security measurements; probability sampling yields unbiased estimates.

The study is the first comprehensive investigation of how sampling strategies affect web security measurement conclusions, comparing datasets and strategies across 500k Tranco domains and 24.8M Common Crawl hosts. It shows Top-N selection does not reflect the overall web distribution and may bias observed vulnerability rates, while probability-based strategies yield stable, unbiased prevalence and impact estimates. Hybrid sampling offers no advantage because its deterministic prefix consistently hurts accuracy, and the authors propose an adaptive probability-based strategy effective even when target prevalence is unknown.

arXiv cs.CR · 6d agoResearch1

Creepy crawlies

git.kernel.org spends more CPU rendering commit pages for abusive AI-era scrapers than on all legitimate access including git clones.

Konstantin Ryabitsev reports that abusive crawler 'background radiation' at git.kernel.org consumes more CPU cycles rendering commit HTML than all legitimate access, with 14 cores across five geo-distributed nodes dedicated to scraper traffic. Simon Willison highlights the implications for crawlable web services like Datasette.

Simon Willison · 8d agoAI industry1

terms.txt: A Consent and Compensation Protocol for Agentic Web Access

terms.txt specifies a robots.txt-style protocol for per-path, per-purpose AI crawler consent and compensation, with enforcement adding 0.20-0.65 ms per request.

The paper documents that automated clients now make up most web requests, that training dominates Cloudflare-classified crawling, and that the largest AI platforms fetch thousands of pages per returned visitor while robots.txt cannot express identity, purpose, terms, or price. It specifies terms.txt plus an origin-enforced exchange using Web Bot Auth signatures, signed intent, delegation tokens, HTTP 402 negotiation, and signed receipts. A dependency-free implementation adds 0.20 to 0.65 ms per request on one vCPU.

arXiv cs.CR · 6d agoResearch

Reflectiz Launches Agentic Pentesting for Websites: Up to 10x Coverage vs Conventional Pentests

Reflectiz launched a multi-agent AI penetration testing platform for websites, claiming up to 10x more coverage than conventional pentests.

Reflectiz, a continuous web exposure management vendor, launched a multi-agent penetration testing platform for websites under a new Offensive Hub. Four specialized agents crawl the site like a real user, fingerprint the stack, run attacks, and independently validate findings to remove false positives. The company claims up to 10x more coverage than conventional pentesting by starting from its pre-existing live model of each site. Findings integrate via REST API, CI/CD triggers and Slack, and the Atlas remediation agent guides fixes.

CSO Online · 8d agoTools

Reflectiz Launches Agentic Pentesting for Websites: Up to 10x Coverage vs Conventional Pentests

Reflectiz launched a multi-agent AI penetration testing platform for websites, claiming up to 10x more coverage than conventional pentests by leveraging existing site context.

Reflectiz, a web exposure management vendor, announced an agentic penetration testing product within a new Offensive Hub. The platform uses specialized AI agents for crawling, stack fingerprinting, attack execution, and independent validation, claiming to cover the OWASP Top 10 and reduce false positives. Findings integrate with existing workflows via REST API, CI/CD triggers, and Slack, and an AI remediation agent named Atlas guides fixes. The announcement is a vendor press release with a planned September 15 webinar demo.

GBHackers · 8d agoTools 2 sources

New Phishing Attack Uses Blob URLs to Hide Malicious Pages From Security Scanners

Barracuda detailed a DocuSign-themed phishing campaign that renders credential-harvesting pages as browser blob URLs, evading URL reputation and blocklist defenses.

Barracuda researchers report a credential-harvesting phishing campaign that starts with a DocuSign-themed email containing a calendar invitation and routes victims through legitimate Microsoft OAuth endpoints and Microsoft Teams. A crafted redirect parameter leads Teams to load external content from cdn.bloom[.]io, which the browser renders as a blob URL — a session-only address held in local memory with no persistent public URL to crawl, categorize, or blocklist. The locally generated page registers a service worker, runs inside a sandboxed iframe, and is dynamically steered by backend infrastructure, indicating a centrally managed phishing platform. Barracuda recommends phishing-resistant MFA such as FIDO2 keys and passkeys, monitoring OAuth flows and redirect chains, and Teams malicious URL protection.

GBHackersupdated · 6d agofirst · 6d agoPhishing & fraud in the wild 3 sources

WordPress Security Plugins: How to Choose the Right One

Sucuri's guide breaks WordPress security plugins into hardening, malware scanning, integrity monitoring, and filtering types, and explains how to evaluate and layer them.

The Sucuri guide explains that WordPress security plugins bundle five capabilities - hardening, malware detection, integrity monitoring, activity logging, and application-level filtering - and that plugins run only after WordPress loads, unlike server-level firewalls. It lists leading causes of compromise: outdated plugins and themes, weak or reused credentials, nulled premium software, insecure configuration, and shared-hosting cross-contamination. It concludes with evaluation criteria and a post-installation security checklist for owners without dedicated security teams.

Sucuri Blog · 11d agoIndustry

CDN Tsunami Attack Abuses HTTP/3 Translation for Up to 350x DoS Amplification

Researchers disclosed two DoS attacks exploiting CDNs' HTTP/3-to-HTTP/1.1 translation to amplify low-bandwidth requests up to 350x against origin servers.

Researchers named the two techniques HTTP/3 Bandwidth Amplification (HBA) and HTTP/3 Connection Amplification (HCA), exploiting the gap where CDNs speak HTTP/3 to browsers but only HTTP/1.1 to origins. All six tested CDNs (Alibaba, Baidu, Cloudflare, CloudFront, Fastly, Tencent) were susceptible to HBA, and five to HCA, with Cloudflare immune to HCA because it buffers full requests before opening backend connections. Attackers using under 5 Mbps forced over 100 Mbps at origins, and the 350x factor applies only to Alibaba, Baidu, and Tencent, which support the QPACK dynamic table. Testing identified 42,330 potentially vulnerable subdomains out of 151,685 hosted on the six providers across the Tranco Top 1M.

The Hacker News · 27d agoResearch

Have it both ways: stay discoverable in search while disallowing AI training

Cloudflare launches Disallow AI Training setting letting sites block AI training via robots.txt while staying indexed in search; Apple, Google, and Microsoft honor it.

Cloudflare announced a 'Disallow AI Training' setting that publishes a no-training preference in robots.txt so sites can block AI training (including by mixed-use crawlers) without losing search indexing. Apple, Google, and Microsoft meet Cloudflare's new 'Accountable' designation, which requires training/summary opt-out mechanisms, URL-level training visibility, and assurance that opt-outs don't affect search rankings. Cloudflare cites that under 1% of its sites block search bots while 17% block AI training, and its Block settings now apply to mixed-use crawlers as well. Granular controls over how much content appears in AI summaries are planned for early next year.

Cloudflare Blog · 1d agoTools