Indirect Instruction Injection in Multi-Modal LLMs
Full article127 words · extracted from schneier.com · click to collapse
Interesting research: “(Ab)using Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs”:
Abstract: We demonstrate how images and sounds can be used for indirect prompt and instruction injection in multi-modal LLMs. An attacker generates an adversarial perturbation corresponding to the prompt and blends it into an image or audio recording. When the user asks the (unmodified, benign) model about the perturbed image or audio, the perturbation steers the model to output the attacker-chosen text and/or make the subsequent dialog follow the attacker’s instruction. We illustrate this attack with several proof-of-concept examples targeting LLaVa and PandaGPT.
Tags: academic papers, AI, LLM, machine learning
Sidebar photo of Bruce Schneier by Joe MacInnis.
Text extracted automatically; images, tables and formatting may be missing. Original: https://www.schneier.com/blog/archives/2023/07/indirect-instruction-injection-in-multi-modal-llms.html