Search: “gpu-management”

29 stories

DomainAll Cyber AIWindow24h 3d 7d 30d AllIn the wild Important only Show duplicates

CategoryAll

SeverityAll critical 0 high 0 medium 0 low 0 info 0

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Chaotic Eclipse Released GreenSection, A PoC For NVIDIA Memory Corruption Zero-Day

Researcher Chaotic Eclipse released GreenSection, a PoC exploiting an unpatched out-of-bounds write in NVIDIA Windows user-mode shared memory.

Security researcher Chaotic Eclipse disclosed an out-of-bounds write in NVIDIA's Windows user-mode components, which share a global memory section (\BaseNamedObjects\{52813408-3561-4705-820a-2b3b78be92ba}) with full read/write access to all users. The PoC crashes applications running Vulkan or OpenGL; the researcher says the bug could cross user boundaries or compromise dwm.exe but did not fully assess the impact. No CVE has been assigned and no patch is mentioned. The researcher recently released zero-day PoCs against Kaspersky, Avast, and CrowdStrike Falcon.

Security AffairsSep 7, 13:42 UTC · 8d agoExploit / PoC 2 sources

Towards Standardized Evaluation of GPU Memory Safety with GMSBench

GMSBench provides 149 CUDA tests covering spatial, temporal, and concurrency GPU memory errors, exposing detection gaps in Compute Sanitizer.

GMSBench is a GPU memory safety benchmark comprising 149 self-contained CUDA tests spanning spatial, temporal, and concurrency errors across different GPU memory spaces and execution scenarios. The authors evaluate NVIDIA's Compute Sanitizer across multiple GPU architectures using the suite, exposing gaps in its detection coverage. The benchmark offers a standardized foundation for comparative evaluation of GPU memory safety mechanisms.

arXiv cs.CRSep 8, 15:11 UTC · 7d agoResearch

GPU-CFR: 80x Faster Counterfactual Regret Minimization by Compiling the Game to Static Dataflow and CUDA Graph Replay

GPU-CFR compiles counterfactual regret minimization into static dataflow with CUDA Graph Replay, achieving 29.8-80.4x speedups over prior GPU solvers.

The paper presents a compiler and runtime that turns any fixed game's CFR iteration into a static dataflow graph of flat arrays and precomputed indices, cutting framework operations by up to 18.1x. Because shapes and buffer addresses never change, CUDA Graph Replay records the iteration once and replays it with a single launch. On one A100 across an eight-game suite, GPU-CFR runs 29.8-80.4x faster than the fastest prior GPU CFR and 14-258x faster than the CPU implementation LiteEFG on the four largest games, while reproducing reference iterates bitwise on CPU.

arXiv cs.AI / cs.LG / cs.CLSep 10, 17:58 UTC · 5d agoAI research3· 1 read

Inside NVIDIA’s cuDNN Graph API: Fusion, Autotuning, and Plan Reuse with cuDNN Frontend

MarkTechPost tutorial walks through NVIDIA's cuDNN Frontend graph API, covering kernel fusion, autotuning, plan reuse, and CUDA graph capture on Colab GPUs.

The tutorial explains how to express GPU computations as operation graphs via the cuDNN Frontend graph API, running the five-step build pipeline of validate, build operation graph, create execution plans, check support, and build plans. It progresses from a single fused convolution with bias and ReLU to autotuning across engine configs, FP8-style epilogues, attention, plan serialization, dynamic shapes, and CUDA graph capture. Each kernel is benchmarked against a PyTorch reference on a single Colab GPU to verify correctness and measure cost. The piece also covers practical setup issues like making libcudnn.so visible to the frontend's dynamic loader.

MarkTechPostSep 15, 21:37 UTC · 15h agoAI tools & infra

Speculative Decoding in vLLM on AMD GPUs

vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.

The vLLM project documents draft-and-verify speculative decoding support for AMD GPUs via ROCm, comparing native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark drafting approaches. Output-token throughput effects varied with drafting method, proposal length, model family, draft checkpoint, workload, and acceptance behavior. The post also covers how to enable each method plus practical tuning and observability considerations.

Hacker News · AISep 7, 09:26 UTC · 9d agoAI tools & infra HN 141↑ · 52 comments1

Implementation of Machine Learning Workflows with NVIDIA cuML, RAPIDS, GPU Benchmarking, Explainability, Clustering, and Model Inference

Hands-on tutorial implements NVIDIA cuML and RAPIDS to GPU-accelerate scikit-learn-style ML workflows with benchmarking, clustering, and inference.

The tutorial demonstrates NVIDIA cuML as a GPU-accelerated machine learning framework, using cuml.accel to speed up unmodified scikit-learn scripts with zero code changes and the native cuML API for CuPy/cuDF interoperability. It benchmarks CPU versus GPU implementations of PCA, K-Means, nearest-neighbor search, logistic regression, random forests, and DBSCAN on datasets up to 200,000 samples with 64 features. It also builds GPU pipelines with UMAP, t-SNE, and HDBSCAN, validates GPU-generated SHAP explanations, uses the FIL library for forest inference, and covers model serialization and GPU/CPU portability.

MarkTechPostSep 13, 01:42 UTC · 3d agoAI tools & infra

Why Scaling AI Compute Performance Requires a New Power Architecture

NVIDIA argues AI factories need 800 VDC power distribution as dense GPU racks outgrow traditional AC-based delivery.

NVIDIA's blog contends each generation of accelerated computing demands higher rack density and more efficient, scalable power distribution. It frames the bottleneck as how power moves from the grid to the GPU rather than raw wattage, and describes limitations of traditional AC power delivery. NVIDIA advocates a new 800 VDC power architecture for AI factories.

NVIDIA BlogAug 11, 15:00 UTC · Aug 11, 2026AI industry

OPEN-1B: A Fully Auditable Training Run

Open-1B releases a 1B-parameter model with bitwise-reproducible training, letting independent auditors verify every step of the run on commodity hardware.

The paper introduces a 'fully auditable' tier of model transparency: every training operation is reproducible with bitwise certainty on heterogeneous commodity hardware by imposing definite ordering on GPU kernel reductions, data batch ordering, and collective communication. Because replaying a full run on one machine is infeasible, a collective verification scheme lets many independent auditors certify individual steps covering the whole run. The authors release Open-1B with its full pretraining dataset, every intermediate checkpoint, the training codebase, and an audit harness. This rules out undisclosed data, injected biases, or backdoors that proof-of-learning or proof-of-training-data techniques cannot exclude.

arXiv cs.AI / cs.LG / cs.CLSep 15, 16:23 UTC · 20h agoAI research

FlashVector: Agent for Hierarchical Model Serving Stack Optimization

FlashVector agent optimizes all layers of Unity's ad-serving stack, delivering up to 2x model-server throughput and 1.98x latency speedup in production.

FlashVector is an agentic system that optimizes performance across GPU kernels, ML framework computation graphs, model servers, and on-demand feature processing. Deployed in Unity's Vector advertising platform, it achieved up to 2x model-server throughput increase, 1.98x latency speedup, and 1.6x feature-store throughput gain. Optimizations spanned NVIDIA Triton's C++ codebase and the Python feature transformation service, demonstrating extensibility beyond single-kernel tuning.

arXiv cs.AI / cs.LG / cs.CLSep 15, 16:27 UTC · 20h agoAI research

Perplexity Details Its GPU Embedding Stack: How Ivy, Tulip and ROSE Serve pplx-embed

Perplexity details its GPU embedding serving stack (Ivy, Tulip, ROSE), which reuses LLM prefill/decode kernels, CUDA graphs, and LazyTensors to cut launch overhead.

Perplexity engineers published a deep dive on the serving infrastructure behind pplx-embed, used across Perplexity Search and its API platform. The stack comprises Ivy (Rust HTTP gateway), Tulip (gRPC scheduling and batching), and ROSE (Runtime-Optimized Serving Engine), which reuses LLM prefill and decode kernels rather than running a separate embedding engine. Optimizations include whole-model CUDA graphs with lazy capture and a LazyTensor abstraction that overlaps CPU batch preparation with in-flight GPU work. Benchmarks are reported against vLLM v0.22.0 in BF16, with FlashAttention 4 generally fastest but FlashInfer 3 winning on Qwen-based models at very long sequence lengths.

MarkTechPostSep 6, 03:20 UTC · 10d agoAI tools & infra1

CUDA for AMD on Windows

GitHub project 'CUDA for AMD on Windows' trends on Hacker News, aiming to run CUDA workloads on AMD GPUs under Windows.

A GitHub repository named CUDA-for-AMD-Windows drew 45 points and 5 comments on Hacker News. The project targets making NVIDIA's CUDA software stack work on AMD GPUs under Windows, which matters to developers running GPU compute or inference on non-NVIDIA hardware. No security impact is described.

Hacker News · AISep 13, 14:25 UTC · 2d agoAI tools & infra HN 45↑ · 5 comments1

Same Cluster, 33 Points More Utilization: What Changed Was the Order

A Dharma AI blog post on Hugging Face claims GPU cluster utilization rose 33 points after changing job ordering.

A community blog post published on Hugging Face, part of a GPU management series by Dharma AI, discusses improving utilization on the same GPU cluster. According to the title, reordering jobs or tasks was the change that produced roughly 33 additional points of utilization. No article body was available for further detail.

Hugging Face BlogAug 17, 19:46 UTC · 29d agoAI tools & infra

Miles v0.1: Production-Level Post-Training

Radix Ark open-sources Miles v0.1, a full-stack RL post-training framework demonstrated with asynchronous agentic RL on GLM-5.2 744B-A40B across 64 GB300 GPUs.

Miles v0.1 is a full-stack, open-source system for frontier-scale reinforcement-learning post-training, built on slime with rollout engines on SGLang and trainers supporting NVIDIA Megatron-LM and PyTorch FSDP backends plus three weight-synchronization transports. It supports full-parameter RL, LoRA RL, on-policy distillation, supervised fine-tuning, true-on-policy rollout-training alignment, and extends to diffusion models. The end-to-end case study ran fully asynchronous agentic RL on GLM-5.2 744B-A40B for terminal-use coding tasks on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. The code is open-sourced on GitHub.

Hugging Face daily papersSep 7, 20:00 UTC · 8d agoAI tools & infra

GPUThor: Amplifying Rowhammer Attacks via Non-Uniform Patterns to Exploit ECC-Protected GPUs

GPUThor uses non-uniform hammering to amplify Rowhammer on NVIDIA GPUs, achieving 500X-23,500X more bit flips and first exploits of ECC-protected GPUs.

GPUThor is a Rowhammer attack on NVIDIA GPUs that reverse-engineers memory-access coalescing behavior to enable non-uniform hammering patterns activating aggressor rows more intensely than decoy rows. By identifying refresh instances where in-DRAM mitigations apply, it constructs longer patterns that escape mitigation across refresh intervals. It yields 500X to 23,500X more bit flips than prior GPU Rowhammer attacks across NVIDIA A4000, A4500, A5000, and A6000 GPUs, and enables the first Rowhammer exploits on ECC-protected GPUs via uncorrectable double and triple bit flips, making denial-of-service and privilege-escalation attacks practical.

arXiv cs.CRSep 15, 02:46 UTC · 1d agoResearch

Chipmaker Patch Tuesday: Nvidia, AMD, Arm Issue Security Advisories

AMD, Arm, and Nvidia issued Patch Tuesday advisories fixing a Linux GPU driver DoS flaw, nine Mali GPU vulnerabilities, and two high-severity Triton defects.

AMD fixed CVE-2026-43603, a NULL pointer dereference in its Linux GPU kernel driver that can crash systems and cause denial-of-service, credited to SecMate researchers, with patches for EPYC, Ryzen, Radeon, and Instinct shipped in July and embedded variants due in October. Arm published an advisory covering nine Mali GPU vulnerabilities allowing use-after-free access, kernel information leaks, or DoS, releasing fixes for Valhall and 5th Gen GPU Architecture drivers, with Bifrost also affected. Nvidia updated Triton Inference Server for Linux to resolve two high-severity flaws, one causing DoS and one enabling information disclosure, data tampering, and DoS. Intel had issued no new advisories since the previous Patch Tuesday.

SecurityWeekSep 9, 16:22 UTC · 6d agoAdvisory CVE-2026-43603

Building a Linux GPU Driver for the M4 Mac Mini in One Month

Two developers built a fully OpenGL ES 3.0 compliant Linux GPU driver for the M4 Mac Mini in one month via clean-room reverse engineering.

Niklas and the author reverse engineered Apple's AGX GPU firmware ABI and user-space components in about a month, a process that normally takes years, producing an OpenGL ES 3.0 conformant driver fast enough to run Minecraft at 200fps on an M4 Mac Mini. The work was done transparently using hypervisor traces without examining Apple binaries, following clean-room practices, and included a custom shader compiler, command stream builder, and a full Linux kernel driver for the firmware ABI. The A18 Pro firmware ABI proved significantly more complex than the M1's, with 1.5x as many structs and twice as many pointers. All experiments and provenance evidence were published in public agx-re repositories.

Hacker News · AISep 15, 19:30 UTC · 17h agoResearch HN 40↑ · 5 comments

How XPUs Meet a World-Class AI Factory

NVIDIA argues AI factories with custom XPUs and NVLink Fusion connectivity must optimize tokens-per-second, tokens-per-watt, cost and uptime.

NVIDIA published a blog explaining that AI factories running continuously are economically defined by delivered output: tokens per second, tokens per watt, cost per token, utilization and uptime. It argues hyperscalers and AI-native companies building custom XPUs need infrastructure designed as a complete factory rather than collections of individual accelerators. The piece promotes NVIDIA's NVLink Fusion and full-stack XPU connectivity as the foundation for such world-class AI factory builds.

NVIDIA BlogAug 24, 15:00 UTC · 22d agoAI industry

SpliTEE: Improving LLM Inference on Trusted Hardware with Differentially Private GPU Outsourcing

SpliTEE splits LLM inference between Intel TDX trusted execution and untrusted GPUs, using differential privacy instead of encryption to protect intermediate representations.

SpliTEE extends split inference to LLMs, running inference partly inside an Intel TDX TEE while masking intermediate inputs sent to untrusted GPUs with differential privacy rather than encryption. The authors show a prompt-reconstruction attack recovers nearly 80% of prompts from unmasked intermediate representations, motivating the masking. A global sensitivity analysis bounds the required DP noise scale, avoiding quantization and keeping models in floating point. The implementation is nearly twice as fast as full CPU-based TDX inference and 5-15 seconds faster than encryption-based Slalom with higher accuracy, evaluated on Llama-3.2-3B and Qwen3-4B.

arXiv cs.CRSep 14, 04:55 UTC · 2d agoResearch

NVIDIA Details BioNeMo Inference Runtime (BioIR): 2.90x Higher Boltz-2 Folding Throughput and 58.5K Residues per GPU-Hour on 8xH100

NVIDIA released BioNeMo Inference Runtime (BioIR), an open-source PyTorch-compatible library delivering 2.90x higher Boltz-2 protein-folding throughput on 8xH100 GPUs.

NVIDIA detailed BioIR, a Python library that accelerates Boltz-2, OpenFold2, and OpenFold3 structure-prediction inference on NVIDIA GPUs while preserving standard PyTorch workflows. On a matched benchmark of 1,000 human dimers on 8xH100 80GB GPUs, BioIR delivered 58.5K folded residues per GPU-hour versus 20.2K for a torch.compile baseline, a 2.90x throughput gain. BioIR already powered the AlphaFold Database expansion, generating about 31 million candidate complexes across 4,777 proteomes, with 1.81 million released as high-confidence predictions. Extrapolated to 1 million targets, estimated folding energy drops from 35 MWh to 11 MWh at 8-GPU TDP equivalents.

MarkTechPostSep 10, 21:57 UTC · 5d agoAI tools & infra1

27.5KB language-agnostic WebGPU syntax highlighter

A developer released gpu-lexer, a 27.5KB language-agnostic syntax highlighter that uses a tiny WebGPU model to label code tokens in the browser.

gpu-lexer splits source into words, whitespace, and symbols, then a small WebGPU model uses local and whole-file context to assign nine token classes, working on languages never seen in training. On held-out files, 12.57% of token labels differ from Shiki, though this measures agreement with Shiki rather than objective correctness. In benchmarks against Shiki 4.4.3, Prism.js, Highlight.js, Sugar High, and Starry Night, it highlighted 10 concatenated copies of three.min.js (5.56M characters) about 10x faster on an Apple M4 Pro in Chrome 152. The author frames it as an experiment, not a grammar-equivalent highlighter.

Hacker News · securitySep 9, 01:14 UTC · 7d agoAI tools & infra HN 21↑ · 5 comments1

Scalability Analysis of Distributed Kolmogorov-Arnold Network Training on High-Performance Computing Systems

An empirical study shows distributed Kolmogorov-Arnold Network training reaches 74.7% parallel efficiency at 8 A100 GPUs, with overheads driven by All-Reduce choices.

The study evaluates data-parallel Kolmogorov-Arnold Network (KAN) training on the FinisTerrae III supercomputer using up to 8 NVIDIA A100 GPUs across 4 nodes with PyTorch Distributed Data Parallel. Strong scaling yields 5.97x speedup and 74.7% parallel efficiency at 8 GPUs, comparable to conventional deep learning workloads, while communication overhead ranges from 1.3% to 6.1%, driven mainly by All-Reduce algorithm selection and inter-node latency rather than KAN's edge-wise gradient structure. Weak scaling shows an initial single-to-multi-GPU throughput drop followed by stability, and the parameter-to-memory ratio improves with model size even as training time scales unfavorably. The authors provide GPU topology and model-size deployment guidelines for KAN training.

arXiv cs.AI / cs.LG / cs.CLSep 7, 16:45 UTC · 8d agoAI research1

NVIDIA Announces CUDA Rust with cuda-oxide (SIMT) and cutile-rs (Tile) for Compile-Time-Safe GPU Kernels

NVIDIA launches CUDA Rust via open-source cuda-oxide (SIMT) and cutile-rs (Tile), bringing compile-time-safe Rust GPU kernels.

NVIDIA announced CUDA Rust, making Rust a first-class language for GPU kernels through two NVlabs open-source projects: cuda-oxide for the SIMT model and cutile-rs for the Tile model. Both use Rust's ownership and borrow checker to catch buffer aliasing bugs at compile time. cutile-rs is published on crates.io, runs on stable Rust 1.89+ with CUDA 13.3, and is already used in Hugging Face's Grout inference engine and mistral.rs; cuda-oxide is early alpha requiring nightly Rust, CUDA 12.x, and compute capability 8.0+. cuda-oxide compiles Rust MIR through the community Pliron IR framework and LLVM to PTX, while cutile-rs JIT-compiles kernels via CUDA Tile IR.

MarkTechPostSep 8, 19:51 UTC · 7d agoAI tools & infra1

Arm Mali G2-Ultra NX GPU: desktop-class mobile gameplay with AI-native graphics

Arm unveiled Mali G2-Ultra NX, its first AI-native mobile GPU with in-shader neural acceleration, third-gen ray tracing, and up to 24% higher benchmark performance.

Arm announced the Mali G2-Ultra NX, the first AI-native Mali GPU, integrating neural accelerators directly into shader cores alongside a new execution engine and third-generation hardware ray tracing. It introduces Neural Super Sampling (NSS), Neural Frame Rate Upscaling (NFRU), and Neural Super Sampling and Denoising (NSSD); the Neural Dawn demo with Sumo Digital showed up to 4x performance efficiency and 70% lower external memory traffic versus native rendering. Arm claims up to 24% higher benchmark performance, 13% lower DRAM traffic on ray tracing benchmarks, and up to 120 FPS with NFRU. Over 14 billion Mali GPUs have shipped to date.

Hacker News · AISep 8, 04:06 UTC · 8d agoAI industry HN 71↑ · 53 comments1

Class Is in Session: GeForce NOW Levels Up Linux, Chromebooks and More

NVIDIA's GeForce NOW cloud gaming service ships its native Linux app out of beta and adds streaming optimizations for Frame Generation responsiveness.

NVIDIA announced that the native Linux application for its GeForce NOW cloud gaming service is officially out of beta. The update also delivers cloud optimizations that make Frame Generation feel more responsive while streaming, and Performance members will see higher frame rates. The announcement is timed for the back-to-school season and also mentions improvements for Chromebooks.

NVIDIA BlogAug 13, 13:00 UTC · Aug 13, 2026AI industry

From Megawatts to Tokens: How NVIDIA Maximizes AI Factory Production

NVIDIA detailed DSX power-management results: Lambda gained 24% token throughput at fixed power, and an AI factory auto-shed 1MW via Emerald AI's grid program.

NVIDIA says Lambda's first validation of DSX MaxLPS on HGX B200 servers ran 19 nodes within a 16-node power budget, lifting cluster token throughput 24% (roughly 4M to 5M tokens/second) and improving performance per watt by 23%. NVIDIA projects DSX MaxLPS can enable up to 40% more GPU capacity for Vera Rubin NVL72 factories within the same megawatt budget. Emerald AI's Conductor platform, running at NVIDIA's Eos factory with Silicon Valley Power, responded to over 200 utility demand signals, automatically dropping power from 4MW to 3MW without interrupting priority workloads. The first dedicated DSX Flex commercial deployment is planned at a 96-megawatt Manassas, Virginia facility.

NVIDIA BlogSep 15, 16:55 UTC · 20h agoAI industry

Nvidia is the central bank of AI

The Economist's interactive briefing argues Nvidia functions as AI's central bank, controlling the compute supply underpinning the industry's growth.

An Economist interactive briefing titled 'Nvidia is the central bank of AI' examines the chipmaker's dominant control over AI compute supply. The link drew 45 points and 23 comments on Hacker News. The listing provides only engagement metrics, with the full analysis hosted on the Economist's site.

Hacker News · securitySep 12, 15:08 UTC · 3d agoAI industry HN 45↑ · 23 comments

Graph Machine: Towards Better Pretraining via Edges

Researchers propose Graph Machine, an O(n)-state sparse architecture that replaces 75% of Qwen3-0.6B dense layers with only slight loss change.

The paper introduces the Graph Machine (GM), an architecture that maintains an O(n)-sized state accessed through sparse, dynamic routing via pointer-like edges updated differentiably by a referral mechanism resembling pointer chasing. The authors replaced 75% of dense Transformer layers in Qwen3-0.6B with GM sparse layers and pretrained from scratch on 15.7B tokens. Retrieving 2 of 4,096 tokens per KV head in each sparse layer degrades loss only slightly, while retrieving 4 marginally improves loss over the dense baseline.

Hugging Face daily papersSep 1, 20:00 UTC · 14d agoAI research

Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI

Hugging Face released @huggingface/kernels, a library offering 200+ WebGPU compute kernels to accelerate AI inference locally in browsers.

Hugging Face introduced the @huggingface/kernels package, bundling more than 200 optimized WebGPU compute kernels for running AI workloads locally. The release targets browser-based and on-device inference, reducing reliance on server-side compute. No article body was available beyond the title, so benchmark results and supported models are not specified.

Hugging Face BlogSep 1, 00:00 UTC · 15d agoAI tools & infra

NVIDIA security advisory (AV26-849)

Canada's Cyber Centre relayed NVIDIA security advisories covering vulnerabilities in NemoClaw, OpenShell, Unified Fabric Manager, and DGX Spark.

The Canadian Centre for Cyber Security issued bulletin AV26-849 flagging NVIDIA vulnerabilities across NemoClaw and OpenShell (multiple versions), NVIDIA Unified Fabric Manager (multiple versions and models), and DGX Spark (versions prior to 1.110.13). The advisory links to NVIDIA security bulletins and urges users and administrators to apply updates as they become available. No exploitation details or CVE identifiers are enumerated in the advisory text.

Canadian Centre for Cyber SecurityAug 26, 14:17 UTC · 20d agoAdvisory