Monet: Measuring the Ecosystem of Open-Source Text-to-Image Models Tailored for Harmful Services
Researchers found 23,947 harmful open-source image models, some exceeding 19 million downloads.
A measurement study identified 23,947 open-source text-to-image models intentionally tailored for harmful services, termed Monets, across eight model hubs and ten policy-based harm categories. The most popular exceeded 19 million downloads, 40.76 percent were mirrored across hubs, and 11.99 percent stayed accessible after bans on their original platforms. Researchers also observed keyword obfuscation, safeguard circumvention, and commercial campaigns, including one spanning 668 models with 914 completed commissions. Downstream distribution through GitHub projects and inference APIs raised child-safety concerns.
- Researchers found 23,947 harmful text-to-image models across eight hubs.
- The most popular model exceeded 19 million downloads.
- 40.76 percent were mirrored, and 11.99 percent remained available after bans.
- One commercial campaign covered 668 models and 914 completed commissions.
- Distribution also occurred through GitHub projects and inference APIs.
- Organizations
- GitHub
From the original article
1 min read at arxiv.orgThe open-source text-to-image (T2I) ecosystem enables rapid model development and sharing, but also hosts models intentionally tailored for harmful services, which we call Monets. Prior work has examined specific types of harmful T2I models on individual platforms, but a Monet does not exist in isolation.
By Zihao Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.