IAMRAGBENCH benchmark shows RAG poisoning risks in LLM-generated AWS IAM policies
Original title: IAMRAGBENCH: Benchmarking Authorization Expansions in LLM-Generated AWS IAM Policies under RAG Poisoning
The 60-case benchmark uncovers 46 goal-matched attacks from poisoned context on five AWS endpoints.
LLMs using RAG synthesize AWS IAM policies from tasks and documents. Attacker-controlled context can produce provider-valid policies exceeding trusted authorization. IAMRAGBENCH evaluates this with deterministic scoring and case-specific validation, finding high rates of authorization deltas and goal-matched violations.
- 60-case benchmark covering scope, conditions, denies, trust, boundaries, and delegation
- 49 exact authorization deltas from included-context exposure
- 46/292 goal-matched attacks at 15.75%
- BM25 retrieves poisons for all tasks at k=5
- Products
- IAM policies
- Organizations
- AWS
- AI models
- LLM
From the original article
1 min read at arxiv.orgLarge language models (LLMs) using retrieval-augmented generation (RAG) can synthesize Amazon Web Services (AWS) Identity and Access Management (IAM) policies from tasks and operational documents. Attacker-controlled context may induce provider-valid policies that exceed trusted authorization requirements.
By Hasan Kaplan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.