ZeroHour
Infosecurity Magazinepublished ()ingested Alessandro Mascellino

NVIDIA Container Toolkit Vulnerability Exposes AI Systems to Risk

criticalVulnerabilityimportance 60CVE-2024-0132

Vulnerabilities mentionedAll →

CVEVulnerabilityCVSSEPSSFlagsAffectedExposurePublished
CVE-2024-0132
TOCTOU Flaw in NVIDIA Container Toolkit Grants Host Filesystem Access from Containers

NVIDIA Container Toolkit versions 1.16.1 and earlier contain a Time-of-Check Time-of-Use (TOCTOU) race condition (CWE-367) that, with the default configuration, allows a specifically crafted container image to gain access to the host file system. The flaw is triggered when a vulnerable toolkit prepares GPU resources for a running container, so environments that execute attacker- or user-supplied images — such as multi-tenant AI platforms and shared GPU clusters — face the greatest risk, while deployments using CDI mode are explicitly not affected. A successful exploit can lead to code execution, escalation of privileges, information disclosure, data tampering, or denial of service on the host. Affected operators include those running the toolkit standalone and those deploying it through the NVIDIA GPU Operator. No confirmed in-the-wild exploitation is documented (not in CISA KEV), but headlines indicate researchers demonstrated working exploits, including one bypassing the initial patch, and EPSS assigns a 40.8% probability of exploitation within 30 days (99th percentile).

Do: Upgrade NVIDIA Container Toolkit to a fixed release newer than 1.16.1 and update NVIDIA GPU Operator deployments so they bundle the fixed toolkit; because researchers demonstrated a bypass of the initial patch, confirm the latest vendor release is installed. Where feasible, switch the container runtime to CDI mode, which is not affected by this flaw, and restrict GPU hosts to running trusted images. Review hosts that have run untrusted or user-submitted images for signs of compromise.

8.341%
  • NVIDIA Container Toolkit 1.16.1 and earlier (default configuration; CDI-mode use is not affected)
  • NVIDIA GPU Operator deployments bundling NVIDIA Container Toolkit 1.16.1 or earlier
masson the order of 1M+ GPU-accelerated host installations, though only a subset run untrusted images (exact counts unknown)
Full article306 words · extracted from infosecurity-magazine.com · click to collapse

A newly discovered vulnerability in the NVIDIA Container Toolkit (CVE-2024-0132) could allow attackers to break out of containerized environments and gain access to sensitive data and systems.

The vulnerability, discovered by Wiz researchers, affects both cloud-based and on-premises AI applications using the toolkit, which is widely used to enable GPU support in containers.

According to Wiz, this exposes organizations to the risk of data breaches, loss of control over infrastructure and potential exposure of customer information.

Shared environments, such as those using Kubernetes, are especially vulnerable, as attackers could access data and resources across entire clusters.

High-Risk Environments

The vulnerability poses significant risks in several scenarios:

  • Single-tenant environments where malicious container images could compromise a user's system
  • Multi-tenant platforms like Kubernetes, where a container breakout could impact other applications sharing the same GPU resources
  • AI service providers, where a breach could expose other customers' data, models, and secrets

NVIDIA's toolkit is a crucial component in enabling GPU usage within containers, which are a cornerstone of modern AI workflows.

Its widespread adoption has made the flaw a significant security concern, particularly for AI developers and cloud service providers.

Read more on container security risks in shared AI environments:  Over Half of Users Report Kubernetes/Container Security Incidents

Wiz researchers noted that this vulnerability underscores the need for more robust security measures in AI infrastructures.

They recommend that organizations do not rely solely on containers for isolation, urging the use of additional layers like virtualization to safeguard sensitive data and workloads.

NVIDIA issued a patch for the vulnerability on September 26 2024.

Affected organizations are urged to update the NVIDIA Container Toolkit to version 1.16.2 and the NVIDIA GPU Operator to version 24.6.2.

This is particularly critical for environments that allow third-party container images or where users may run untrusted AI models.

Image credit: Juan Roballo / Shutterstock.com

Text extracted automatically; images, tables and formatting may be missing. Original: https://www.infosecurity-magazine.com/news/nvidia-ai-toolkit-vulnerability/