ActGov: Governing LLM Agent Actions via Policy-Constrained Validation
ActGov validates each LLM agent tool call at runtime, cutting indirect prompt-injection success while preserving task utility.
Researchers present ActGov, a runtime framework that validates every LLM-proposed tool action before it has external effects. ActGov-Policy iteratively builds policies from tool specifications, benign tasks, and failure traces, verifying each update with SMT-based counterexample checking. At runtime, a call is allowed only if it stays inside the task-scoped authorization boundary and satisfies applicable policies. On AgentDojo and AgentDyn, across multiple models and attack setups, ActGov reduced indirect prompt-injection success while preserving task utility and outperformed existing defenses.
- Per-action runtime checks enforce task-scoped authorization on dynamic agent workflows.
- Policies are iteratively built and verified with SMT counterexample checking.
- Evaluated on AgentDojo and AgentDyn against indirect prompt injection.
- Attack success fell while task utility was preserved versus prior defenses.
Full article210 words · extracted from arxiv.org · click to collapse
Large language model (LLM) agents increasingly execute long-horizon workflows through external tools, allowing untrusted outputs to influence subsequent actions and exceed user authorization. Existing defenses isolate injected content or constrain execution with predefined plans and static policies, but these approaches are brittle under dynamic workflows and scale poorly across extensible tool ecosystems. In this work, we present ActGov, a runtime enforcement framework that validates each LLM-proposed tool action before it causes external effects. Built on a unified semantic model of authorization, actions, runtime context, and security constraints, the ActGov-Policy component iteratively constructs a policy set from tool specifications, benign tasks, and observed failure traces, with each update verified through SMT-based counterexample checking. At runtime, ActGov-Runtime abstracts each tool call into finite policy records and permits it only if it remains within the task-scoped authorization boundary and satisfies all applicable policies. This per-action enforcement preserves authorization throughout long-horizon, dynamically branching workflows. We evaluate ActGov on the AgentDojo and AgentDyn benchmarks across multiple models and attack configurations. It shows that ActGov consistently reduces the success rate of indirect prompt-injection attacks while preserving task utility, significantly outperforming existing defenses. These results demonstrate that ActGov can enforce fine-grained authorization over dynamic agent executions without relying on the underlying LLM to correctly identify malicious instructions.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.24446