Monet: Measuring the Ecosystem of Open-Source Text-to-Image Models Tailored for Harmful Services
Researchers found 23,947 harmful open-source image models, some exceeding 19 million downloads.
A measurement study identified 23,947 open-source text-to-image models intentionally tailored for harmful services, termed Monets, across eight model hubs and ten policy-based harm categories. The most popular exceeded 19 million downloads, 40.76 percent were mirrored across hubs, and 11.99 percent stayed accessible after bans on their original platforms. Researchers also observed keyword obfuscation, safeguard circumvention, and commercial campaigns, including one spanning 668 models with 914 completed commissions. Downstream distribution through GitHub projects and inference APIs raised child-safety concerns.
- Researchers found 23,947 harmful text-to-image models across eight hubs.
- The most popular model exceeded 19 million downloads.
- 40.76 percent were mirrored, and 11.99 percent remained available after bans.
- One commercial campaign covered 668 models and 914 completed commissions.
- Distribution also occurred through GitHub projects and inference APIs.
Full article214 words · extracted from arxiv.org · click to collapse
The open-source text-to-image (T2I) ecosystem enables rapid model development and sharing, but also hosts models intentionally tailored for harmful services, which we call Monets. Prior work has examined specific types of harmful T2I models on individual platforms, but a Monet does not exist in isolation. The broader Monet ecosystem, spanning model characteristics, cross-platform propagation, governance evasion, monetization, and downstream deployment, remains poorly understood. In this study, we present the first systematic, ecosystem-level measurement of Monets. Grounded in the policies of real-world model hubs, we construct a taxonomy of ten harmful service categories and identify 23,947 Monets across eight major T2I model hubs, with the most popular exceeding 19 million downloads. While some developers employ anti-theft mechanisms against unauthorized re-uploading, Monets propagate across platforms at scale, with 40.76% mirrored across hubs. Such propagation further enables governance evasion via cross-platform archiving, keeping 11.99% of Monets accessible after bans on their original platforms, alongside other evasion strategies including keyword obfuscation and model-level safeguard circumvention. Monets also anchor coordinated commercial campaigns---one spanning 668 models with 914 completed commissions and another advertising gray-market account-farming service---and reach users through GitHub projects and inference APIs, raising downstream child safety concerns. These findings expose the limitations of platform-siloed defenses and highlight the need for cross-platform threat intelligence, coordinated governance, and technical safeguards.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.24134