What happens when AI agent governance is missing at scale

In this interview with Help Net Security, Gourab Basu, Global Head of Engineering at meshIQ, discusses governance in AI agent systems. He argues that instructions written into a prompt are not enough to control what an agent does, since agents can change their own path as they work. Real control means checking proposed actions before they reach production systems, such as pausing a large refund for human approval.

He also covers what breaks when a company scales from ten agents to a thousand, and how to build governance that works across different agent frameworks.

What’s a lesson from integration work that most people building agent systems today have not learned yet, because they haven’t been burned by it?

One lesson integration engineers learned a long time ago is that orchestration and control cannot be the same thing. Traditional integration workflows are generally deterministic: engineers define the path on top of the underlying connectors, and the workflow executes according to that design. AI agents introduce a nondeterministic orchestration layer. Give an agent a goal, and it can decide which tools to use, in what order, and how to adjust its path as the workflow unfolds.

That dynamism is powerful, but it also means the prompt cannot become the control boundary. Integration engineers stopped blindly trusting callers decades ago. Assuming an agent will always follow an instruction in a prompt is the modern equivalent of trusting a partner system to always send perfectly formed XML. If the execution path can change dynamically, governance has to follow that path and operate inside the flow if you want control over what happens.

Give me a real example: an agent proposes an action, and a control sitting inside the execution path stops it before it reaches a production system. What was the action, and what would have happened if it had gone through?

Consider a refund workflow with a simple business rule: refunds below $100 can proceed automatically, while anything above $100 requires human confirmation. In a traditional integration workflow, that condition can be explicitly designed into the execution path. With an AI agent, if the same rule exists only in the prompt, it remains an instruction.

An in-flow governance layer can inspect the proposed tool call and its parameters before the refund executes. A refund below the threshold can proceed, while one above $100 is paused until a human confirms it. The important distinction is timing: the policy is enforced while the action can still be stopped, rather than discovering afterward that an agent issued a refund outside the approved threshold. That is what turns governance from observation into control.

Say a company goes from ten agents to a thousand agents in a year. What in your experience is the first thing that quietly falls apart if governance was not built into the flow from the start?

The first thing you lose is confidence that changes to enterprise systems of record are being governed consistently. With ten agents, teams may still understand which agents have access to which systems and compensate with manual oversight. At a thousand, the number of agents capable of changing enterprise records grows dramatically, and more capable models can increase the scope of the actions those agents attempt.

The problem is that most enterprise controls were designed around human users and traditional applications. Those safeguards still matter, but relying on governance only at the destination system is not enough when an intelligent agent is making dynamic decisions about what action to take next. Governance has to become part of the agent execution flow so the rules governing an action remain consistent before it reaches the system of record.

That does not make a nondeterministic agent deterministic. It creates predictable boundaries around nondeterministic execution. And at scale, policy also has to determine which actions can proceed automatically and which genuinely require human intervention; otherwise, manual review either becomes a bottleneck or gets bypassed.

Can you describe rebuilding or migrating a governance layer so it worked across frameworks instead of inside just one? What had to change, and what stayed the same?

To make governance work across agent frameworks, the architecture has to be separated into loosely coupled layers. At the center is a framework-independent governance engine. The policy model, decision semantics—allow, flag or confirm, escalate, and block or deny—the audit-record format, evaluation suite, human-in-the-loop controls, and policy authoring should not change simply because an organization changes frameworks.

What does change is the interception mechanism. Different frameworks expose tool execution at different points in their lifecycle: one may use callbacks, another an on_call_tool mechanism such as FastMCP, while another may expose a raw tool loop. Some may not provide a reliable pre-execution interception point at all. The adapter has to account for those differences and establish the point at which governance can participate in execution.

That separation is important because governance should span the agent architecture. Enterprises will use different frameworks for different purposes and those choices will change. The policies governing what agents are permitted to do should remain consistent even when the underlying technology does not.

If a younger engineer asked you how to avoid the mistakes you made scaling autonomous systems, what’s the one story you’d tell them first?

The lesson I would give them is not to scale capability faster than control. AI is an extraordinarily powerful technology, and it is easy to focus first on everything an autonomous system can do. At small scale, teams can often compensate for weak controls through manual review and close supervision. That becomes a risky bet at enterprise scale.

Before scaling autonomous systems, think through control, observability, and governance as part of the architecture itself. Understand what an agent can access, what actions it can take, where human judgment is required, and how its decisions will be reconstructed afterward. Those capabilities become much harder to retrofit once agents are already embedded across production workflows. The measure of progress is not simply how much autonomy you can introduce, but how confidently you can govern that autonomy as it expands.

Download: eBook: Identity-First Threat Intelligence