NVIDIA wants AI agent safety enforced in silicon, not left to the agent
NVIDIA released an open platform that confines AI agents with runtime policies and separate-hardware monitoring.
NVIDIA introduced the Open Agent Safety Platform, pairing OpenShell software with a hardware reference design called Sentry. OpenShell runs agents in isolation and enforces policies on files, networks, processes, and other resources, with low overhead on Vera CPUs and the ability to extend to Arm and Intel systems. Sentry, using NVIDIA DOCA on BlueField-4 DPUs, monitors agent activity independently and can quarantine an agent that crosses its software boundary. Scale AI said it is adopting the reference design for mission-critical enterprise and government agents, and NVIDIA published the software on GitHub.
- OpenShell isolates agents and enforces file, network, and process policies.
- Sentry on BlueField-4 DPUs can quarantine agents that cross boundaries.
- Open-source OpenShell targets Vera CPUs and can extend to Arm and Intel.
- Scale AI is using the design for enterprise and government systems.
Full article522 words · extracted from helpnetsecurity.com · click to collapse
NVIDIA has introduced an open software platform and a hardware-based reference design intended to keep AI agents within the limits set by the organizations that use them.
The Open Agent Safety Platform includes OpenShell, software that controls what an agent can access, and Sentry, a watchdog in the reference design that monitors agent activity from separate hardware. Organizations can deploy elements of the platform according to their needs.
NVIDIA Open Agent Safety Platform Reference Design (Source: NVIDIA)
NVIDIA said reports of AI agents escaping test environments and accessing systems without permission helped drive the platform’s development. The incidents prompted debate about whether AI-agent development should slow down.
According to the company, agents can also stray from assigned tasks when they encounter a blocked action, a bug, a missing tool or undefined instructions. Long-running tasks add to that risk, making independent security controls necessary.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” said Jensen Huang, CEO of NVIDIA. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering. NVIDIA Open Agent Safety Platform brings together industry, researchers and public-sector organizations to share best practices, align on evaluation methods and foster international cooperation. Together, we can raise the bar for global AI safety.”
Controls across three layers
NVIDIA describes the platform as three layers that work together as an agent carries out a task. The application layer includes the AI model, tools and data the agent uses. The runtime layer places the agent on computing systems, monitors its activity and enforces access rules. Beneath it, the infrastructure layer provides the hardware, network connections and other resources needed to run the agent.
OpenShell operates at the runtime layer. It runs agents in isolated environments and enforces policies governing access to files, networks, processes and other resources while agents work. NVIDIA says OpenShell runs with minimal overhead on its Vera CPUs and, as open-source software, can be extended to other computing platforms, including those from Arm and Intel.
Monitoring agents from separate hardware
Sentry adds controls at the infrastructure layer through NVIDIA’s BlueField-4 data processing units. Using NVIDIA’s DOCA software, it monitors agent activity and enforces access policies. If an agent attempts to cross its software boundary, Sentry can quarantine and stop it.
In NVIDIA’s Vera Rubin POD design, BlueField-4 sits on the path an agent uses to reach its AI model. That position allows Sentry to monitor activity independently of the agent. Sentry is part of the platform’s reference system design and requires supported hardware.
“Scale AI is using the NVIDIA Open Agent Safety Platform reference design to build reliable agentic AI systems for our enterprise and government customers running mission-critical applications, with isolation, policy enforcement and auditability built in from the start,” said Francis deSouza, CEO of Scale AI. “We support agentic security with clear boundaries that define what agents can do, and controls that keep them operating within those permissions.”
NVIDIA said the platform’s software, including OpenShell and related skills, is available through its developer resources and GitHub.