ActGov: Governing LLM Agent Actions via Policy-Constrained Validation
ActGov validates each LLM agent tool call at runtime, cutting indirect prompt-injection success while preserving task utility.
Researchers present ActGov, a runtime framework that validates every LLM-proposed tool action before it has external effects. ActGov-Policy iteratively builds policies from tool specifications, benign tasks, and failure traces, verifying each update with SMT-based counterexample checking. At runtime, a call is allowed only if it stays inside the task-scoped authorization boundary and satisfies applicable policies. On AgentDojo and AgentDyn, across multiple models and attack setups, ActGov reduced indirect prompt-injection success while preserving task utility and outperformed existing defenses.
- Per-action runtime checks enforce task-scoped authorization on dynamic agent workflows.
- Policies are iteratively built and verified with SMT counterexample checking.
- Evaluated on AgentDojo and AgentDyn against indirect prompt injection.
- Attack success fell while task utility was preserved versus prior defenses.
From the original article
1 min read at arxiv.orgLarge language model (LLM) agents increasingly execute long-horizon workflows through external tools, allowing untrusted outputs to influence subsequent actions and exceed user authorization. Existing defenses isolate injected content or constrain execution with predefined plans and static policies, but these approaches are brittle under dynamic workflows and scale poorly across extensible tool ecosystems.
By Kaiyuan Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.