Stop watching what AI agents say and start watching what they do

In this interview with Help Net Security, Ariel Assaraf, CEO of Coralogix, explains why a system prompt can describe a boundary for an AI agent but cannot enforce one. Assaraf covers how his team builds AI agent guardrails at the execution layer, limits context without expanding authority, and tunes controls by risk so agents stay useful during incidents. He describes how he tracks the consequences of agent actions, since an agent can return 200s and still do harm.

You’ve drawn a line between telling an agent what it shouldn’t do and preventing it from doing it. Walk us through a real incident where that distinction mattered: where a documented policy or system prompt existed, but the agent still crossed the line. What was missing technically?

The recent Gemini hacking incident is a good example of this. The agent was supposed to operate inside a controlled cybersecurity test, but a configuration error gave it internet access and it entered three real systems. It eventually recognized the mistake and stopped, which is encouraging, but by then the boundary had already been crossed.

The missing controls were outside the model: network isolation, target allowlists, scoped credentials and an independent authorization check before execution. Those controls need to exist regardless of what the model has been instructed to do.

A system prompt can describe a boundary. It cannot enforce one. If an agent has the technical ability to cross that boundary, enterprises have to assume that at some point it might. The enforcement mechanism needs to sit between the agent and the action it is attempting to take.

How do you distinguish between an agent that needs more context to perform well and an agent that’s being handed too much context, increasing its attack surface and its ability to justify a bad action to itself?

More context should improve the agent’s knowledge, not expand its authority.

We try to provide the minimum context required for the current step, retrieved just in time, with clear provenance and expiration. Sensitive or untrusted context should be separated from instructions, and access to information should never automatically grant permission to act on it. An agent knowing something and an agent being authorized to do something with that information are two different things.

The practical test is measurable: does additional context improve task success in evaluations, and what does it do to policy violations, prompt-injection success, and unnecessary data access? If context adds little performance but materially increases those risks, it should not be there.

The goal is not to give an agent everything it could possibly need. It is to give it what it needs for the task in front of it, while keeping authority independently constrained.

What does a hard guardrail look like at the infrastructure or execution layer, as opposed to a prompt-level instruction? Can you describe one you built and how you tested that it stops the agent?

A hard guardrail is one the model cannot reason around.

The pattern we use is a policy enforcement point between the agent and its tools. Every proposed action is evaluated against the agent’s identity, target, environment, data classification, and current authorization. The agent never receives unrestricted credentials, so even if it decides that an action is appropriate, it cannot simply execute outside of those controls.

A read-only production query might be allowed automatically. A write operation may require approval. A destructive action can be blocked entirely. The important distinction is that the model is not the final authority over whether its own action is permitted.

We test this with evaluations, prompt injections, obfuscated commands, and multi-step attempts designed to reach the same prohibited outcome through different paths. Success means the action produces no external side effect and the block itself generates a complete telemetry record. Beyond knowing that the guardrail worked, you need to know what the agent attempted and why it was stopped.

Tell us about a guardrail that was too aggressive and broke legitimate agent workflows. How did you tune it, and what tradedoff did you land on between safety and usefulness?

The guardrail that becomes too aggressive the fastest is requiring human approval for every production-related action. It is safe, but it also stops agents from doing basic investigation and makes them nearly useless during incidents. If a human has to approve every read or routine diagnostic step, much of the value of autonomy disappears.

The better approach is to classify actions by risk and state transition. Read-only queries within an approved scope can run automatically. Reversible changes receive tighter checks. Irreversible or high-blast-radius actions require human approval.

That also means looking at what an action can actually change, rather than treating every interaction with a production environment as equally dangerous.

The tradeoff is not safety vs usefulness. It is about applying friction in proportion to potential impact. The higher the potential consequence of an action, the stronger the control should be.

Traditional monitoring is built around uptime, latency, and error rates. An agent can be running perfectly, returning 200s, and still making terrible decisions. What metrics or signals have you built to catch that specific category of failure?

The most important signal is not the model’s response. It is the consequence of the action it takes.

We correlate agent telemetry with the rest of the organization’s observability data: tool calls, policy decisions, overrides, retries and data access alongside deployments, incidents, security events, rollbacks, and human corrections. Looking at those signals together makes it possible to see when an agent is technically functioning as designed but producing an operationally bad outcome.

We also measure decision consistency. If the same observable facts produce different actions across model versions, we need to see where the policy path diverged. A change in model behavior should not become an invisible change in production behavior.

This is not “chain of thought” monitoring, but an external decision record showing which policy was applied, what evidence was evaluated, which exception was used, and what happened next. That distinction is important because enterprises do not need access to a model’s internal reasoning to establish accountability. They need a reliable record of the inputs, permissions, actions and consequences they can actually observe.

An agent can return perfect 200s while creating a terrible outcome. So the outcome is what must be observable.

Download: The IT leader’s guide to AI code sprawl