Project Analyzing Human Language Usage Shuts Down Because ‘Generative AI Has Polluted the Data’
Full article239 words · extracted from 404media.co · click to collapse
🖥️
404 Media is an independent website whose work is written, reported, and owned by human journalists and whose intended audience is real people, not AI scrapers, bots, or a search algorithm. Sign up to support our work and for free access to this article. Learn why we require this here.
The creator of an open source project that scraped the internet to determine the ever-changing popularity of different words in human language usage says that they are sunsetting the project because generative AI spam has poisoned the internet to a level where the project no longer has any utility.
Wordfreq is a program that tracked the ever-changing ways people used more than 40 different languages by analyzing millions of sources across Wikipedia, movie and TV subtitles, news articles, books, websites, Twitter, and Reddit. The system could be used to analyze changing language habits as slang and popular culture changed and language evolved, and was a resource for academics who study such things. In a note on the project’s GitHub, creator Robyn Speer wrote that the project “will not be updated anymore.”
This post is for paid members only
Become a paid member for unlimited ad-free access to articles, bonus podcast content, and more.
Sign up for free access to this post
Free members get access to posts like this one along with an email round-up of our week's stories.
Already have an account? Sign in
Text extracted automatically; images, tables and formatting may be missing. Original: https://www.404media.co/project-analyzing-human-language-usage-shuts-down-because-generative-ai-has-polluted-the-data/