ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing
ACEA is a pluggable arena scoring LLM red-team attackers and blue-team defenses head-to-head with an LLM judge and verifiable leakage ground truth.
ACEA connects pluggable red- and blue-team adapters to a shared target LLM through the model-agnostic ASAP HTTP protocol and scores attack and defense rates per adversarial round. Canonical seeded secrets provide verifiable ground truth that separates real leakage from hallucination, and attacks are delivered to the target even when blocked to measure raw potency. The platform adds real-time battle visualization, failure-localizing end-of-battle reports, and an optional in-context improvement loop that feeds advisory hints between rounds.
- ASAP protocol lets any-language adapters participate without persistent state
- Per-round decomposition separates attack strength from defense effectiveness
- Seeded secrets give ground truth filtering out judge hallucination
- Advisory hints enable cross-round adapter improvement without state
- AI models
- ACEA
From the original article
1 min read at arxiv.orgAutomated red-team attacks and blue-team defenses for large language models (LLMs) are advancing quickly. However, attackers and defenders are built and tested in isolation, and the resulting scores are hard to trust.
By Yi Ting Shen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.