AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety
AgentSafeEvo co-trains attacker and defender agents to harden LLM tool use against adversarial tasks.
AgentSafeEvo is a co-evolutionary reinforcement-learning framework for LLM agent safety. An attacker builds adversarial tool-use tasks while a defender learns to finish those tasks safely, covering malicious content in user queries, tool descriptions, and environment observations. Learnability-guided task weighting adjusts each task's training contribution from within-task defender-reward variability. Experiments on agent-safety and general-capability benchmarks report improved robustness without sacrificing ordinary agent performance.
- Attacker and defender co-evolve across diverse agent tasks and attack types.
- Attacks can arrive via queries, tool descriptions, or environment observations.
- Learnability-guided weighting uses defender-reward variability to scale task updates.
- Reported result is stronger adversarial robustness without losing general capability.
From the original article
1 min read at huggingface.co
Large language model (LLM) agents accomplish complex tasks through multi-turn interactions with external tools. However, malicious content embedded in user queries, tool descriptions, or environment observations can manipulate their decisions and induce unsafe actions.
By Chenxu Wang, Xiaoyu Wen, Han Qi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.