Microsoft and Reddit Are Fighting About Why Bing’s Crawler Is Blocked on Reddit
Full article216 words · extracted from 404media.co · click to collapse
Microsoft and Reddit are offering conflicting explanations for why Microsoft’s search engine, Bing, is currently blocked from crawling Reddit and offering links from the site in its search results.
Reddit, which now demands payment from anyone crawling the site and using its data to train AI products, claims that Bing’s crawler is being used to power AI products. Microsoft claims it has made it easy for any site to block its crawler that’s used for AI products, while still allowing a crawler that is only used for search results, and that Reddit’s decision to block Bing is “impacting competition” in the search engine space.
The conflicting reasonings behind the block are further proof that the massive, indiscriminate scraping of the internet to create AI training data in a way that violates long-respected norms about how to access information on the web are eroding trust, making the internet less open, and causing tech companies to beef about this issue in public.
This post is for paid members only
Become a paid member for unlimited ad-free access to articles, bonus podcast content, and more.
Sign up for free access to this post
Free members get access to posts like this one along with an email round-up of our week's stories.
Already have an account? Sign in
Text extracted automatically; images, tables and formatting may be missing. Original: https://www.404media.co/microsoft-and-reddit-are-fighting-about-why-bings-crawler-is-blocked-on-reddit/