ZeroHour

Search: “gpu-kernels”

30 stories

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Implementation of Machine Learning Workflows with NVIDIA cuML, RAPIDS, GPU Benchmarking, Explainability, Clustering, and Model Inference

Hands-on tutorial implements NVIDIA cuML and RAPIDS to GPU-accelerate scikit-learn-style ML workflows with benchmarking, clustering, and inference.

The tutorial demonstrates NVIDIA cuML as a GPU-accelerated machine learning framework, using cuml.accel to speed up unmodified scikit-learn scripts with zero code changes and the native cuML API for CuPy/cuDF interoperability. It benchmarks CPU versus GPU implementations of PCA, K-Means, nearest-neighbor search, logistic regression, random forests, and DBSCAN on datasets up to 200,000 samples with 64 features. It also builds GPU pipelines with UMAP, t-SNE, and HDBSCAN, validates GPU-generated SHAP explanations, uses the FIL library for forest inference, and covers model serialization and GPU/CPU portability.

MarkTechPost · 3d agoAI tools & infra

Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI

Hugging Face released @huggingface/kernels, a library offering 200+ WebGPU compute kernels to accelerate AI inference locally in browsers.

Hugging Face introduced the @huggingface/kernels package, bundling more than 200 optimized WebGPU compute kernels for running AI workloads locally. The release targets browser-based and on-device inference, reducing reliance on server-side compute. No article body was available beyond the title, so benchmark results and supported models are not specified.

Hugging Face Blog · 15d agoAI tools & infra

Inside NVIDIA’s cuDNN Graph API: Fusion, Autotuning, and Plan Reuse with cuDNN Frontend

MarkTechPost tutorial walks through NVIDIA's cuDNN Frontend graph API, covering kernel fusion, autotuning, plan reuse, and CUDA graph capture on Colab GPUs.

The tutorial explains how to express GPU computations as operation graphs via the cuDNN Frontend graph API, running the five-step build pipeline of validate, build operation graph, create execution plans, check support, and build plans. It progresses from a single fused convolution with bias and ReLU to autotuning across engine configs, FP8-style epilogues, attention, plan serialization, dynamic shapes, and CUDA graph capture. Each kernel is benchmarked against a PyTorch reference on a single Colab GPU to verify correctness and measure cost. The piece also covers practical setup issues like making libcudnn.so visible to the frontend's dynamic loader.

MarkTechPost · 10h agoAI tools & infra

Towards Standardized Evaluation of GPU Memory Safety with GMSBench

GMSBench provides 149 CUDA tests covering spatial, temporal, and concurrency GPU memory errors, exposing detection gaps in Compute Sanitizer.

GMSBench is a GPU memory safety benchmark comprising 149 self-contained CUDA tests spanning spatial, temporal, and concurrency errors across different GPU memory spaces and execution scenarios. The authors evaluate NVIDIA's Compute Sanitizer across multiple GPU architectures using the suite, exposing gaps in its detection coverage. The benchmark offers a standardized foundation for comparative evaluation of GPU memory safety mechanisms.

arXiv cs.CR · 7d agoResearch

Perplexity Details Its GPU Embedding Stack: How Ivy, Tulip and ROSE Serve pplx-embed

Perplexity details its GPU embedding serving stack (Ivy, Tulip, ROSE), which reuses LLM prefill/decode kernels, CUDA graphs, and LazyTensors to cut launch overhead.

Perplexity engineers published a deep dive on the serving infrastructure behind pplx-embed, used across Perplexity Search and its API platform. The stack comprises Ivy (Rust HTTP gateway), Tulip (gRPC scheduling and batching), and ROSE (Runtime-Optimized Serving Engine), which reuses LLM prefill and decode kernels rather than running a separate embedding engine. Optimizations include whole-model CUDA graphs with lazy capture and a LazyTensor abstraction that overlaps CPU batch preparation with in-flight GPU work. Benchmarks are reported against vLLM v0.22.0 in BF16, with FlashAttention 4 generally fastest but FlashInfer 3 winning on Qwen-based models at very long sequence lengths.

MarkTechPost · 10d agoAI tools & infra1

Scalability Analysis of Distributed Kolmogorov-Arnold Network Training on High-Performance Computing Systems

An empirical study shows distributed Kolmogorov-Arnold Network training reaches 74.7% parallel efficiency at 8 A100 GPUs, with overheads driven by All-Reduce choices.

The study evaluates data-parallel Kolmogorov-Arnold Network (KAN) training on the FinisTerrae III supercomputer using up to 8 NVIDIA A100 GPUs across 4 nodes with PyTorch Distributed Data Parallel. Strong scaling yields 5.97x speedup and 74.7% parallel efficiency at 8 GPUs, comparable to conventional deep learning workloads, while communication overhead ranges from 1.3% to 6.1%, driven mainly by All-Reduce algorithm selection and inter-node latency rather than KAN's edge-wise gradient structure. Weak scaling shows an initial single-to-multi-GPU throughput drop followed by stability, and the parameter-to-memory ratio improves with model size even as training time scales unfavorably. The authors provide GPU topology and model-size deployment guidelines for KAN training.

arXiv cs.AI / cs.LG / cs.CL · 8d agoAI research1

GPU-CFR: 80x Faster Counterfactual Regret Minimization by Compiling the Game to Static Dataflow and CUDA Graph Replay

GPU-CFR compiles counterfactual regret minimization into static dataflow with CUDA Graph Replay, achieving 29.8-80.4x speedups over prior GPU solvers.

The paper presents a compiler and runtime that turns any fixed game's CFR iteration into a static dataflow graph of flat arrays and precomputed indices, cutting framework operations by up to 18.1x. Because shapes and buffer addresses never change, CUDA Graph Replay records the iteration once and replays it with a single launch. On one A100 across an eight-game suite, GPU-CFR runs 29.8-80.4x faster than the fastest prior GPU CFR and 14-258x faster than the CPU implementation LiteEFG on the four largest games, while reproducing reference iterates bitwise on CPU.

arXiv cs.AI / cs.LG / cs.CL · 5d agoAI research3· 1 read

USN-8760-1: Linux kernel (NVIDIA) vulnerabilities

Canonical issues USN-8760-1 patching multiple Linux kernel (NVIDIA) vulnerabilities across numerous subsystems and CPU architectures.

Canonical published USN-8760-1 to correct several Linux kernel (NVIDIA) vulnerabilities that an attacker could possibly use to compromise the system. The update fixes flaws spanning UAPI, the kernel build system, ARM32, ARM64, RISC-V, S390, and x86 architectures. Patched subsystems also include the block layer, cryptographic API, Compute Acceleration Framework, Intel NPU driver, ACPI and Bluetooth drivers, and the hardware random number generator core.

Ubuntu Security Notices · 22h agoAdvisory

Speculative Decoding in vLLM on AMD GPUs

vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.

The vLLM project documents draft-and-verify speculative decoding support for AMD GPUs via ROCm, comparing native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark drafting approaches. Output-token throughput effects varied with drafting method, proposal length, model family, draft checkpoint, workload, and acceptance behavior. The post also covers how to enable each method plus practical tuning and observability considerations.

NVIDIA Announces CUDA Rust with cuda-oxide (SIMT) and cutile-rs (Tile) for Compile-Time-Safe GPU Kernels

NVIDIA launches CUDA Rust via open-source cuda-oxide (SIMT) and cutile-rs (Tile), bringing compile-time-safe Rust GPU kernels.

NVIDIA announced CUDA Rust, making Rust a first-class language for GPU kernels through two NVlabs open-source projects: cuda-oxide for the SIMT model and cutile-rs for the Tile model. Both use Rust's ownership and borrow checker to catch buffer aliasing bugs at compile time. cutile-rs is published on crates.io, runs on stable Rust 1.89+ with CUDA 13.3, and is already used in Hugging Face's Grout inference engine and mistral.rs; cuda-oxide is early alpha requiring nightly Rust, CUDA 12.x, and compute capability 8.0+. cuda-oxide compiles Rust MIR through the community Pliron IR framework and LLVM to PTX, while cutile-rs JIT-compiles kernels via CUDA Tile IR.

MarkTechPost · 7d agoAI tools & infra1

Building a Linux GPU Driver for the M4 Mac Mini in One Month

Two developers built a fully OpenGL ES 3.0 compliant Linux GPU driver for the M4 Mac Mini in one month via clean-room reverse engineering.

Niklas and the author reverse engineered Apple's AGX GPU firmware ABI and user-space components in about a month, a process that normally takes years, producing an OpenGL ES 3.0 conformant driver fast enough to run Minecraft at 200fps on an M4 Mac Mini. The work was done transparently using hypervisor traces without examining Apple binaries, following clean-room practices, and included a custom shader compiler, command stream builder, and a full Linux kernel driver for the firmware ABI. The A18 Pro firmware ABI proved significantly more complex than the M1's, with 1.5x as many structs and twice as many pointers. All experiments and provenance evidence were published in public agx-re repositories.

OPEN-1B: A Fully Auditable Training Run

Open-1B releases a 1B-parameter model with bitwise-reproducible training, letting independent auditors verify every step of the run on commodity hardware.

The paper introduces a 'fully auditable' tier of model transparency: every training operation is reproducible with bitwise certainty on heterogeneous commodity hardware by imposing definite ordering on GPU kernel reductions, data batch ordering, and collective communication. Because replaying a full run on one machine is infeasible, a collective verification scheme lets many independent auditors certify individual steps covering the whole run. The authors release Open-1B with its full pretraining dataset, every intermediate checkpoint, the training codebase, and an audit harness. This rules out undisclosed data, injected biases, or backdoors that proof-of-learning or proof-of-training-data techniques cannot exclude.

arXiv cs.AI / cs.LG / cs.CL · 15h agoAI research

USN-8748-1: Linux kernel (NVIDIA) vulnerabilities

Ubuntu released USN-8748-1 patching multiple Linux kernel (NVIDIA) vulnerabilities across networking, file system, and crypto subsystems.

Ubuntu issued security notice USN-8748-1 fixing several Linux kernel vulnerabilities affecting the NVIDIA Tegra memory controller driver, hardware crypto device drivers, and network drivers, among others. Affected subsystems include GFS2, OCFS2, and SMB file systems, B.A.T.M.A.N. meshing, Ceph, Netfilter, Open vSwitch, IPv4/IPv6, RxRPC, SCTP, and TIPC. An attacker could potentially exploit these flaws to compromise the system.

Why Scaling AI Compute Performance Requires a New Power Architecture

NVIDIA argues AI factories need 800 VDC power distribution as dense GPU racks outgrow traditional AC-based delivery.

NVIDIA's blog contends each generation of accelerated computing demands higher rack density and more efficient, scalable power distribution. It frames the bottleneck as how power moves from the grid to the GPU rather than raw wattage, and describes limitations of traditional AC power delivery. NVIDIA advocates a new 800 VDC power architecture for AI factories.

NVIDIA Blog · Aug 11, 2026AI industry

GPUThor: Amplifying Rowhammer Attacks via Non-Uniform Patterns to Exploit ECC-Protected GPUs

GPUThor uses non-uniform hammering to amplify Rowhammer on NVIDIA GPUs, achieving 500X-23,500X more bit flips and first exploits of ECC-protected GPUs.

GPUThor is a Rowhammer attack on NVIDIA GPUs that reverse-engineers memory-access coalescing behavior to enable non-uniform hammering patterns activating aggressor rows more intensely than decoy rows. By identifying refresh instances where in-DRAM mitigations apply, it constructs longer patterns that escape mitigation across refresh intervals. It yields 500X to 23,500X more bit flips than prior GPU Rowhammer attacks across NVIDIA A4000, A4500, A5000, and A6000 GPUs, and enables the first Rowhammer exploits on ECC-protected GPUs via uncorrectable double and triple bit flips, making denial-of-service and privilege-escalation attacks practical.

arXiv cs.CR · 1d agoResearch

Miles v0.1: Production-Level Post-Training

Radix Ark open-sources Miles v0.1, a full-stack RL post-training framework demonstrated with asynchronous agentic RL on GLM-5.2 744B-A40B across 64 GB300 GPUs.

Miles v0.1 is a full-stack, open-source system for frontier-scale reinforcement-learning post-training, built on slime with rollout engines on SGLang and trainers supporting NVIDIA Megatron-LM and PyTorch FSDP backends plus three weight-synchronization transports. It supports full-parameter RL, LoRA RL, on-policy distillation, supervised fine-tuning, true-on-policy rollout-training alignment, and extends to diffusion models. The end-to-end case study ran fully asynchronous agentic RL on GLM-5.2 744B-A40B for terminal-use coding tasks on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. The code is open-sourced on GitHub.

Hugging Face daily papers · 8d agoAI tools & infra

CUDA for AMD on Windows

GitHub project 'CUDA for AMD on Windows' trends on Hacker News, aiming to run CUDA workloads on AMD GPUs under Windows.

A GitHub repository named CUDA-for-AMD-Windows drew 45 points and 5 comments on Hacker News. The project targets making NVIDIA's CUDA software stack work on AMD GPUs under Windows, which matters to developers running GPU compute or inference on non-NVIDIA hardware. No security impact is described.

Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training

NVIDIA researchers detail an end-to-end system for online draft co-training that speeds speculative decoding in large-scale long-context RL post-training.

The paper tackles scaling online draft co-training for speculative decoding in RL post-training, where rollout generation dominates cost. It extends packed, load-balanced zigzag ring attention to merge rank-local branch attention with causal main-sequence attention for context parallelism, and introduces TapChannel to transport target features across pipeline-parallel stages without changing the schedule. Experiments show co-trained drafts tracking the policy baseline with substantial rollout and end-to-end speedups up to 122B parameters and strong scaling at 256K tokens.

Hugging Face daily papers · 9d agoAI research

NVIDIA Details BioNeMo Inference Runtime (BioIR): 2.90x Higher Boltz-2 Folding Throughput and 58.5K Residues per GPU-Hour on 8xH100

NVIDIA released BioNeMo Inference Runtime (BioIR), an open-source PyTorch-compatible library delivering 2.90x higher Boltz-2 protein-folding throughput on 8xH100 GPUs.

NVIDIA detailed BioIR, a Python library that accelerates Boltz-2, OpenFold2, and OpenFold3 structure-prediction inference on NVIDIA GPUs while preserving standard PyTorch workflows. On a matched benchmark of 1,000 human dimers on 8xH100 80GB GPUs, BioIR delivered 58.5K folded residues per GPU-hour versus 20.2K for a torch.compile baseline, a 2.90x throughput gain. BioIR already powered the AlphaFold Database expansion, generating about 31 million candidate complexes across 4,777 proteomes, with 1.81 million released as high-confidence predictions. Extrapolated to 1 million targets, estimated folding energy drops from 35 MWh to 11 MWh at 8-GPU TDP equivalents.

MarkTechPost · 5d agoAI tools & infra1

MathKernel: An evidence-aware multi-engine mathematics kernel and MCP server

MathKernel is an open-source, evidence-aware multi-engine mathematics kernel that exposes verification workflows to AI agents via an MCP server.

MathKernel, published on GitHub, is a mathematics kernel that combines multiple computation engines with evidence-aware outputs. It ships as an MCP server, enabling AI agents and coding assistants to perform and verify calculations. The project drew moderate attention on Hacker News.

27.5KB language-agnostic WebGPU syntax highlighter

A developer released gpu-lexer, a 27.5KB language-agnostic syntax highlighter that uses a tiny WebGPU model to label code tokens in the browser.

gpu-lexer splits source into words, whitespace, and symbols, then a small WebGPU model uses local and whole-file context to assign nine token classes, working on languages never seen in training. On held-out files, 12.57% of token labels differ from Shiki, though this measures agreement with Shiki rather than objective correctness. In benchmarks against Shiki 4.4.3, Prism.js, Highlight.js, Sugar High, and Starry Night, it highlighted 10 concatenated copies of three.min.js (5.56M characters) about 10x faster on an Apple M4 Pro in Chrome 152. The author frames it as an experiment, not a grammar-equivalent highlighter.

USN-8728-1: Linux kernel (GCP) vulnerabilities

Ubuntu issued kernel security update USN-8728-1 for GCP kernels fixing Arm TLB and AMD Zen 2 privilege escalation flaws (CVE-2025-10263, CVE-2025-54518).

Ubuntu released USN-8728-1, a security update for the Linux kernel used on Google Cloud Platform images. It fixes CVE-2025-10263, where certain Arm processors complete broadcast TLB invalidation before memory writes are globally observed, allowing local attackers to bypass memory protections or escalate privileges, and CVE-2025-54518, an AMD Zen 2 operation cache isolation flaw that can corrupt higher-privilege instructions. The notice also corrects several other kernel security issues.

MaxKernel: Agentic Kernel Generation for TPUs

Researchers open-source MaxKernel, a multi-agent LLM system that generates and optimizes TPU kernels matching expert hand-tuned baselines on JaxBench.

MaxKernel is a multi-agent system offering three paradigms for TPU kernel development: human-in-the-loop collaborative design, a fully autonomous metric/trace-driven optimization loop, and graph-based autonomous search for global exploration. All paradigms draw on a shared pool of specialized sub-agents for planning, implementation, self-debugging, testing, and hardware profiling. Evaluated on JaxBench's 50 diverse TPU kernel tasks and real-world workloads from open-source models, it consistently matches expert hand-tuned baselines. The system is open-sourced via the AI-Hypercomputer GitHub repository.

Hugging Face daily papers · 13d agoAI tools & infra

Arm Mali G2-Ultra NX GPU: desktop-class mobile gameplay with AI-native graphics

Arm unveiled Mali G2-Ultra NX, its first AI-native mobile GPU with in-shader neural acceleration, third-gen ray tracing, and up to 24% higher benchmark performance.

Arm announced the Mali G2-Ultra NX, the first AI-native Mali GPU, integrating neural accelerators directly into shader cores alongside a new execution engine and third-generation hardware ray tracing. It introduces Neural Super Sampling (NSS), Neural Frame Rate Upscaling (NFRU), and Neural Super Sampling and Denoising (NSSD); the Neural Dawn demo with Sumo Digital showed up to 4x performance efficiency and 70% lower external memory traffic versus native rendering. Arm claims up to 24% higher benchmark performance, 13% lower DRAM traffic on ray tracing benchmarks, and up to 120 FPS with NFRU. Over 14 billion Mali GPUs have shipped to date.

How XPUs Meet a World-Class AI Factory

NVIDIA argues AI factories with custom XPUs and NVLink Fusion connectivity must optimize tokens-per-second, tokens-per-watt, cost and uptime.

NVIDIA published a blog explaining that AI factories running continuously are economically defined by delivered output: tokens per second, tokens per watt, cost per token, utilization and uptime. It argues hyperscalers and AI-native companies building custom XPUs need infrastructure designed as a complete factory rather than collections of individual accelerators. The piece promotes NVIDIA's NVLink Fusion and full-stack XPU connectivity as the foundation for such world-class AI factory builds.

NVIDIA Blog · 22d agoAI industry

FlashVector: Agent for Hierarchical Model Serving Stack Optimization

FlashVector agent optimizes all layers of Unity's ad-serving stack, delivering up to 2x model-server throughput and 1.98x latency speedup in production.

FlashVector is an agentic system that optimizes performance across GPU kernels, ML framework computation graphs, model servers, and on-demand feature processing. Deployed in Unity's Vector advertising platform, it achieved up to 2x model-server throughput increase, 1.98x latency speedup, and 1.6x feature-store throughput gain. Optimizations spanned NVIDIA Triton's C++ codebase and the Python feature transformation service, demonstrating extensibility beyond single-kernel tuning.

arXiv cs.AI / cs.LG / cs.CL · 15h agoAI research

Class Is in Session: GeForce NOW Levels Up Linux, Chromebooks and More

NVIDIA's GeForce NOW cloud gaming service ships its native Linux app out of beta and adds streaming optimizations for Frame Generation responsiveness.

NVIDIA announced that the native Linux application for its GeForce NOW cloud gaming service is officially out of beta. The update also delivers cloud optimizations that make Frame Generation feel more responsive while streaming, and Performance members will see higher frame rates. The announcement is timed for the back-to-school season and also mentions improvements for Chromebooks.

NVIDIA Blog · Aug 13, 2026AI industry

Chaotic Eclipse Released GreenSection, A PoC For NVIDIA Memory Corruption Zero-Day

Researcher Chaotic Eclipse released GreenSection, a PoC exploiting an unpatched out-of-bounds write in NVIDIA Windows user-mode shared memory.

Security researcher Chaotic Eclipse disclosed an out-of-bounds write in NVIDIA's Windows user-mode components, which share a global memory section (\BaseNamedObjects\{52813408-3561-4705-820a-2b3b78be92ba}) with full read/write access to all users. The PoC crashes applications running Vulkan or OpenGL; the researcher says the bug could cross user boundaries or compromise dwm.exe but did not fully assess the impact. No CVE has been assigned and no patch is mentioned. The researcher recently released zero-day PoCs against Kaspersky, Avast, and CrowdStrike Falcon.

Security Affairs · 8d agoExploit / PoC 2 sources

Chipmaker Patch Tuesday: Nvidia, AMD, Arm Issue Security Advisories

AMD, Arm, and Nvidia issued Patch Tuesday advisories fixing a Linux GPU driver DoS flaw, nine Mali GPU vulnerabilities, and two high-severity Triton defects.

AMD fixed CVE-2026-43603, a NULL pointer dereference in its Linux GPU kernel driver that can crash systems and cause denial-of-service, credited to SecMate researchers, with patches for EPYC, Ryzen, Radeon, and Instinct shipped in July and embedded variants due in October. Arm published an advisory covering nine Mali GPU vulnerabilities allowing use-after-free access, kernel information leaks, or DoS, releasing fixes for Valhall and 5th Gen GPU Architecture drivers, with Bifrost also affected. Nvidia updated Triton Inference Server for Linux to resolve two high-severity flaws, one causing DoS and one enabling information disclosure, data tampering, and DoS. Intel had issued no new advisories since the previous Patch Tuesday.

SecurityWeek · 6d agoAdvisoryCVE-2026-43603

From Megawatts to Tokens: How NVIDIA Maximizes AI Factory Production

NVIDIA detailed DSX power-management results: Lambda gained 24% token throughput at fixed power, and an AI factory auto-shed 1MW via Emerald AI's grid program.

NVIDIA says Lambda's first validation of DSX MaxLPS on HGX B200 servers ran 19 nodes within a 16-node power budget, lifting cluster token throughput 24% (roughly 4M to 5M tokens/second) and improving performance per watt by 23%. NVIDIA projects DSX MaxLPS can enable up to 40% more GPU capacity for Vera Rubin NVL72 factories within the same megawatt budget. Emerald AI's Conductor platform, running at NVIDIA's Eos factory with Silicon Valley Power, responded to over 200 utility demand signals, automatically dropping power from 4MW to 3MW without interrupting priority workloads. The first dedicated DSX Flex commercial deployment is planned at a 96-megawatt Manassas, Virginia facility.

NVIDIA Blog · 15h agoAI industry