Google’s new agent security system detects tool misuse, loops and rogue behavior

Google’s Agent Anomaly Detection is a reasoning-based oversight and audit layer for autonomous agents deployed on Agent Runtime in the Gemini Enterprise Agent Platform and built with the Agent Development Kit (ADK) for Python 1.2 or later. Google recommends ADK 2.1.0 or later. It is available in Private Preview.

What Agent Anomaly Detection monitors

Agent Anomaly Detection evaluates traces emitted by agents to determine whether they are operating outside their intended boundaries. It flags behavioral anomalies, suspicious intent, and policy violations.

Each anomaly finding includes a severity level, a plain-language explanation of what triggered it, and recommended actions. Findings are published to Security Command Center for triage alongside other security findings.

Agent Anomaly Detection includes detectors for selected risks from the OWASP agentic Top 10: tool misuse, identity and privilege abuse, agentic cascading failures, and rogue agents.

Tool misuse includes risks such as unsafe tool chaining, parameter manipulation, and indirect prompt injection. Identity and privilege abuse includes unauthorized actions arising from issues such as dynamic trust delegation, persona forgery, memory escalation, and confused-deputy vulnerabilities. Agentic cascading failures include infinite execution loops, oscillating retries, fault propagation, and feedback-loop amplification. Rogue-agent detection covers agents that abandon their declared roles, bypass guardrails, or deviate from system instructions.

The system also detects operational risks such as resource exhaustion and escalating token usage.

“We are actively working on the ability for users to define what anomalies mean in the context of their business. This will enable users to write flexible anomaly detectors in natural language together with deterministic rules, which flag when agents operate beyond enterprise-specific business guidelines. In addition, users will be able to validate the accuracy of their new custom business logic on past traffic,” Achuth Narayan Rajagopal, Senior Software Engineer at Google, explained.

Requirements and setup

Several prerequisites must be met to enable Agent Anomaly Detection through one-click provisioning.

Logging and observability buckets must be in the same US multi-region. OpenTelemetry tracing and logging must be enabled through the ADK, raw telemetry must capture prompt inputs and response outputs, and enable_tracing must not be explicitly set to false.

Regional scanner service accounts must have sufficient read access to the logging and observability buckets. Log Analytics and Observability Analytics must also be enabled for the log bucket. Agents must have active telemetry data flow so that their configuration can be verified and they can be enrolled.

Discovered agents appear as monitored agents. Their logs and traces are not analyzed, and no costs are incurred until they are explicitly enabled. Agents that do not meet the requirements are not discovered.

How anomalies are detected

To balance detection speed, cost, and coverage, traces and logs are analyzed in layers. A lightweight first pass scans all traffic for statistical anomalies and flags unusual sessions. An LLM-based reasoning layer then examines those sessions in depth.

For example, consider an Inventory Agent with a list_inventory tool. A user asks to see 100 items at a time, and the agent repeatedly calls the tool with different offsets to retrieve the entire catalog.

The agent may produce no errors or explicit policy violations. Its activity can still represent an unusual usage pattern. The first layer identifies the session as a statistical outlier based on call volume and repetition. The second examines the full exchange, identifies the large-batch, offset-jumping pattern as systematic scraping, and returns a verdict with a plain-language explanation. Where a case needs closer examination, a third, invocation-level layer analyzes individual tool executions, execution states, and parameter histories within the conversation trace.

Session anomaly analysis (Source: Google)

In this example, the system generates a Resource exhaustion finding with Critical severity and a 95% probability. The finding includes its rationale and recommended actions, such as rate-limiting or blocking the list_inventory tool for the user, restricting bulk inventory access through authorization checks, and alerting on large-offset pagination patterns.

Detection runs asynchronously outside the live execution path, so it does not delay the request being analyzed. Applications can retrieve anomaly findings through an API and use an ADK callback or plugin to compare severity and probability against configured thresholds, allowing them to block subsequent tool calls or halt a later turn when those thresholds are exceeded.