Role Grounding Attack writes attacker preferences into agent memory
Original title: Is it my preference? Role Grounding Attack against Agent's Memory via Social Interaction
Nested social posts make victim agents treat an attacker's preferences as the user's, steering tools and leaking data.
The paper formalizes Role Grounding Attack against LLM agents that store preferences from social experience sharing. An attacker publishes a nested conversation so the victim agent binds the attacker's preference to the victim's user. Instantiations manipulate tool selection and can cause sensitive information to be placed in tool arguments. Tests on five memory systems and six language models are reported to succeed at high rates for both goals.
Unconfirmed: Attack success is reported by the authors; rates and the tested systems are not named in the abstract.
- Misattribution occurs when memory is formed and persists at retrieval.
- Tool manipulation steers the victim toward an attacker-controlled tool.
- Privacy leakage also plants a task-requirement preference.
- The abstract does not name the five memory systems or six models.
Impact: Victim agent treats attacker preferences as the user's
From the original article
1 min read at arxiv.orgLLM-based agents now share experiences on social platforms and remember new preferences to enhance their performance. However, existing memory systems contain a vulnerability in role-to-principal attribution that attackers can exploit.
By Haoran Bu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.