LoRango: It Takes Two LoRAs to Unlock Hidden Behaviors in Diffusion Models
LoRango pairs two individually benign LoRA adapters so co-loading unlocks hidden malicious behaviors in Stable Diffusion v1.5 and SDXL with ~98% success.
Researchers present LoRango, a pair-conditioned attack on text-to-image diffusion models where a Signature adapter and a Payload adapter each appear benign alone but release programmed actions when co-loaded together. Matched-pair attack success reaches 97.9% on SD v1.5 and 98.7% on SDXL, versus 2.8-4.6% when adapters are loaded individually. Both adapters export as ordinary static LoRA files requiring no prompt trigger or base-pipeline modification. The work shows individual-adapter inspection is insufficient for multi-LoRA personalization and motivates auditing adapter compositions.
- Signature and Payload LoRA pair triggers programmed actions only when co-loaded with matched partner
- 97.9% attack success on SD v1.5 and 98.7% on SDXL; only 2.8-4.6% when loaded individually
- Adapters are standard static LoRA files needing no prompt trigger or pipeline modification
- Individual-adapter safety review is insufficient; compositions must be audited
Full article198 words · extracted from arxiv.org · click to collapse
Users commonly combine multiple Low-Rank Adaptation (LoRA) adapters to personalize images with different subjects, styles, and visual attributes. Yet inspecting adapters individually does not establish the safety of their composition. We identify and characterize a pair-conditioned attack in text-to-image diffusion: individually useful and benign-appearing adapters redirect image generation when co-loaded with a specifically matched partner, whose identity serves as the trigger. We introduce LoRango to realize this attack through complementary Signature and Payload adapters. The Signature writes a pair-specific code into intermediate carrier representations, while the Payload uses code-selective responses and opposing signal/reference branches. These branches approximately cancel for standalone adapters and mismatched pairs; matched code-reader alignment breaks cancellation within native GEGLU blocks and releases the programmed action. Both adapters are exported as ordinary static LoRA files compatible with standard loaders, requiring no prompt trigger or base-pipeline modification. LoRango achieves matched-pair attack success rates of 97.9\% on SD v1.5 and 98.7\% on SDXL, compared with 2.8--4.6\% when implanted adapters are loaded individually. Further experiments evaluate pair selectivity, standalone fidelity, robustness to deployment variations, and applicability across denoiser architectures. These findings show that individual-adapter inspection is insufficient to assess the security of multi-LoRA personalization and motivate auditing adapter compositions.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.25884