CoSA study shows composed agent skills raise policy-violation rates
Original title: CoSA: The Conflict of Skills Attack in Real-World LLM Agent Ecosystems
Skill additions capped at 5% of each pool raised violations from 35.02% to 58.45% across five models.
Researchers describe Conflict of Skills, where artifacts or signals from one agent skill are reused without the restrictions required for a later use. Their Conflict of Skills Attack adds locally admitted skills to supply missing operations or reshape artifact exchange while leaving existing skills and runtime permissions unchanged. On CoSA-Bench, 288 tasks across recruiting, finance, IT, and business, additions capped at 5% of each pool raised the policy-violation rate from 35.02% to 58.45%. A harness-level guard cut OpenClaw recruiting violations from 65.28% to 6.67%.
Unconfirmed: Results come from controlled executions, not confirmed attacks on production agent deployments.
- Four recurring mechanisms were identified using public skills and single-skill controls.
- The bench covers data leakage, unauthorized state changes, and unsafe execution.
- Tests used five LLMs and three agent harnesses.
- On matched benign controls, compliant completion moved from 66.67% to 65.00%.
- Trace analysis links new violations to cross-skill dependencies involving the additions.
From the original article
1 min read at arxiv.orgAgent Skills allow LLM agents to combine reusable capabilities, but checking each Skill in isolation cannot rule out policy violations when they are composed. We term such failures Conflict of Skills (CoS): artifacts or assessment signals produced for one purpose may be reused downstream without preserving the restrictions or authorization needed for their new use.
By Yujia Zheng · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.