30
30
30
30
NVIDIA Details BioNeMo Inference Runtime (BioIR): 2.90x Higher Boltz-2 Folding Throughput and 58.5K Residues per GPU-Hour on 8xH100
NVIDIA released BioNeMo Inference Runtime (BioIR), an open-source PyTorch-compatible library delivering 2.90x higher Boltz-2 protein-folding throughput on 8xH100 GPUs.
NVIDIA detailed BioIR, a Python library that accelerates Boltz-2, OpenFold2, and OpenFold3 structure-prediction inference on NVIDIA GPUs while preserving standard PyTorch workflows. On a matched benchmark of 1,000 human dimers on 8xH100 80GB GPUs, BioIR delivered 58.5K folded residues per GPU-hour versus 20.2K for a torch.compile baseline, a 2.90x throughput gain. BioIR already powered the AlphaFold Database expansion, generating about 31 million candidate complexes across 4,777 proteomes, with 1.81 million released as high-confidence predictions. Extrapolated to 1 million targets, estimated folding energy drops from 35 MWh to 11 MWh at 8-GPU TDP equivalents.
25
30
30
30
Rapidly scaling online storage to serve over 1 billion ChatGPT users
OpenAI's Habitat online storage platform now handles over 70 million requests per second and 500 PB of data for 1 billion users.
OpenAI details the evolution of Habitat, its online storage platform backing ChatGPT and other products, which began in mid-2024 as a Python client-side library over Azure Cosmos DB. Habitat now processes more than 70 million requests per second, serves over 500 petabytes of data across nearly 40 geographic regions, and supports over 1 billion users weekly. By mid-2025 the client library approach became brittle, so OpenAI moved Habitat into a standalone service to centralize deployments, observability, and multi-tenancy reliability. This is part one of a two-part series; a future post will cover read optimization and scaling the Azure Cosmos DB partnership.
30
27.5KB language-agnostic WebGPU syntax highlighter
A developer released gpu-lexer, a 27.5KB language-agnostic syntax highlighter that uses a tiny WebGPU model to label code tokens in the browser.
gpu-lexer splits source into words, whitespace, and symbols, then a small WebGPU model uses local and whole-file context to assign nine token classes, working on languages never seen in training. On held-out files, 12.57% of token labels differ from Shiki, though this measures agreement with Shiki rather than objective correctness. In benchmarks against Shiki 4.4.3, Prism.js, Highlight.js, Sugar High, and Starry Night, it highlighted 10 concatenated copies of three.min.js (5.56M characters) about 10x faster on an Apple M4 Pro in Chrome 152. The author frames it as an experiment, not a grammar-equivalent highlighter.
15
The VMs Powering Mobile Agents (Instinct, Claude Code)
A teardown reveals Claude Code runs in Firecracker microVMs with a Rust PID 1 and MITM'd egress, while Instinct rents E2B sandboxes with git-based memory.
The author inspects the virtual machines hosting cloud agents: Claude Code runs in a Firecracker microVM with a custom Rust init (process_api) as PID 1, a 324 MB Bun harness on a read-only disk, and 443-only MITM'd SSE egress to api.anthropic.com with host-rotated OAuth tokens and no inbound access. Instinct rents E2B sandbox-as-a-service Firecracker microVMs (Ubuntu 22.04, 2 vCPU, 1.9 GB RAM) where agent memory is a git repo of Markdown committed by the agent and pushed to S3 as a single bundle, using short-lived STS credentials. Both platforms rely on Firecracker, differing mainly in fleet operator and guest boot configuration.
35
Speculative Decoding in vLLM on AMD GPUs
vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.
The vLLM project documents draft-and-verify speculative decoding support for AMD GPUs via ROCm, comparing native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark drafting approaches. Output-token throughput effects varied with drafting method, proposal length, model family, draft checkpoint, workload, and acceptance behavior. The post also covers how to enable each method plus practical tuning and observability considerations.
40
Ask HN: How do you manage skills files?
A Hacker News thread debates whether agent skill files are worth managing, citing 2–4x output-token reductions on flagship models in one company's testing.
Commenters argue skills are stored prompts that help less-technical users compensate for weak prompting, while one participant reports company testing found skills reduce flagship-model output tokens by roughly 2–4x, a gap growing with newer models. Others note skills can bundle reusable scripts and inline commands for deterministic context building, and that harnesses now execute backticked commands before the agent sees the skill. Some argue improving model capability makes downloadable skills redundant.
15
The Evolution of the Agent Harness
Latent Space essay argues late-2025 agent gains came from models and harnesses maturing together, with harness logic absorbed into model weights.
The piece defines the agent harness as everything beyond model weights—tools, context, memory, guardrails—and charts its evolution from ReAct prompting (October 2022) through AutoGPT's premature autonomy, Cursor/Copilot's human-in-the-loop retreat, and Devin's roughly 15% success rate, to o1's capability overhang and Claude Code's February 2025 terminal agent with permission rules. It argues the Christmas 2025 jump cited by Transformer co-inventor Lukasz Kaiser reflected model and harness curves crossing, and that remaining harnesses will serve human attention rather than the model.
32
Synthesized builds Test Data Agent to validate AI agents with production-like data
Synthesized announced a Test Data Agent that provisions production-like data and system states to validate enterprise AI agents before deployment.
Synthesized unveiled its Test Data Agent, an agentic infrastructure capability that generates, masks, and subsets production-representative data for testing AI agents under realistic enterprise conditions. It integrates with agent development, evaluation, testing, and orchestration frameworks, with purpose-built support for complex SAP estates including finance, procurement, and supply-chain workflows and ECC-to-S/4HANA transformation programs. The product runs in on-premises, private-cloud, and hybrid environments and exposes REST APIs and CI/CD triggers for repeatable validation scenarios.
25
30
30
30
30
30
30
30
30
30
30
30
30
30
30
30
30
16 governance tools for securing your AI fleet
CSO Online reviews 16 AI governance and security tools, including Collibra, Credo AI, F5/CalypsoAI, Fiddler AI, and Guardrails AI, for managing LLM risks.
CSO Online surveys 16 vendors in the emerging AI governance and guardrails market for keeping production LLMs in check. Featured products include Collibra's AI Command Center, Confident Security's OpenPCC, Credo AI's Govern AI Assistant, F5's acquired CalypsoAI, Fiddler AI's control plane, and Guardrails AI's Snowglobe simulator. The tools address hallucination tracking, PII leakage, prompt injection and jailbreak defense, and compliance with frameworks such as the EU AI Act, SOC2, ISO-42001, and GDPR.
28
Show HN: Sunk Cost – How long until a local LLM rig pays for itself?
Show HN tool 'Sunk Cost' calculates when a local LLM rig breaks even versus falling API prices, factoring electricity cost and inference speed.
A Hacker News Show HN project called Sunk Cost models the payback period of buying local LLM hardware instead of paying API prices. Users can adjust assumptions like electricity cost ($/kWh) and API speed (tokens/second), and the model assumes API prices keep falling. Where local speed is unmeasured, it is estimated from memory bandwidth divided by bytes read per token, and labelled as an estimate.
12
Implementation of Machine Learning Workflows with NVIDIA cuML, RAPIDS, GPU Benchmarking, Explainability, Clustering, and Model Inference
Hands-on tutorial implements NVIDIA cuML and RAPIDS to GPU-accelerate scikit-learn-style ML workflows with benchmarking, clustering, and inference.
The tutorial demonstrates NVIDIA cuML as a GPU-accelerated machine learning framework, using cuml.accel to speed up unmodified scikit-learn scripts with zero code changes and the native cuML API for CuPy/cuDF interoperability. It benchmarks CPU versus GPU implementations of PCA, K-Means, nearest-neighbor search, logistic regression, random forests, and DBSCAN on datasets up to 200,000 samples with 64 features. It also builds GPU pipelines with UMAP, t-SNE, and HDBSCAN, validates GPU-generated SHAP explanations, uses the FIL library for forest inference, and covers model serialization and GPU/CPU portability.
14
Anthropic Adds Plugin Evals to Claude Code: 6 Grader Types, a No-Plugin Baseline, and a CI Gate for Skills
Anthropic ships a plugin evals workflow for Claude Code with six grader types, a no-plugin baseline arm, and a CI gate via threshold and cost flags.
Anthropic published a plugin evals workflow for Claude Code, exposed via the "claude plugin eval" command on v2.1.269+. Six grader types exist: regex, tool_used, tool_order, and file_exists are free transcript checks, while llm and baseline invoke a billed judge model. Every case runs with and without the plugin, and the delta (Δ) isolates the plugin's contribution; a Δ near zero with a failing tool_used:Skill grader indicates the skill never triggers. CI gating uses --threshold 0.8, --max-cost-usd, --trust-plugin, and --no-publish flags, with results written to a report.html under evals/results/.
42
RTK reports token savings, but our cost benchmarks disagree
Quesma's $1,500 benchmark found RTK cuts reported token output but changes Claude Code and DeepSeek coding costs by only about 5% on Terminal-Bench 2.1.
Quesma benchmarked RTK (Rust Token Killer), a popular tool with 79k GitHub stars that filters terminal output for AI coding agents, whose README claims up to 90% output reduction. Across 1,740 Terminal-Bench 2.1 attempts running Claude Code with Fable 5.0 and OpenCode with DeepSeek V4 Pro 0813, total costs moved only -5% for Fable and +5% for DeepSeek, with pass rates dropping 1-2%. RTK's own rtk gain metric reported 349.2 million tokens saved (an 89% reduction) across 445 DeepSeek attempts, but this did not correlate with actual cost savings, and cached terminal-output reads cost as little as 1/10 to 1/30 of regular input tokens. A bug in rtk find 0.45.0 caused one agent to loop with 339 consecutive errors, costing roughly 9x the baseline attempt, though the task still passed.
42
Miles v0.1: Production-Level Post-Training
Radix Ark open-sources Miles v0.1, a full-stack RL post-training framework demonstrated with asynchronous agentic RL on GLM-5.2 744B-A40B across 64 GB300 GPUs.
Miles v0.1 is a full-stack, open-source system for frontier-scale reinforcement-learning post-training, built on slime with rollout engines on SGLang and trainers supporting NVIDIA Megatron-LM and PyTorch FSDP backends plus three weight-synchronization transports. It supports full-parameter RL, LoRA RL, on-policy distillation, supervised fine-tuning, true-on-policy rollout-training alignment, and extends to diffusion models. The end-to-end case study ran fully asynchronous agentic RL on GLM-5.2 744B-A40B for terminal-use coding tasks on 64 NVIDIA GB300 GPUs with a median step time of 263 seconds over the first 30 measured steps. The code is open-sourced on GitHub.
45
The OCUDU dApp Platform: An Open Runtime and E3 Interface for Real-Time AI-RAN
OCUDU open runtime lets third-party signed AI-RAN dApps run inside production 5G distributed units under three timing contracts, released as BSD-3 preview.
The OCUDU dApp platform provides an open runtime and E3 interface for executing signed AI-RAN applications inside a production 3GPP NR distributed unit, where prior dApp frameworks could only observe export streams. Modules run under three typed timing contracts: GPU receive-chain residency (Class A), the scheduler's 100 microsecond deadline (Class B), or non-blocking observer (Class C). On a GB10 gNB, dApps including an out-of-tree neural equalizer ran on a live cell without fallback. Platform, SDK, and a zero-hardware quickstart are public under BSD-3-Clause-Clear as a preview of the OCUDU AI-RAN Working Group 2.
22
Perplexity Details Its GPU Embedding Stack: How Ivy, Tulip and ROSE Serve pplx-embed
Perplexity details its GPU embedding serving stack (Ivy, Tulip, ROSE), which reuses LLM prefill/decode kernels, CUDA graphs, and LazyTensors to cut launch overhead.
Perplexity engineers published a deep dive on the serving infrastructure behind pplx-embed, used across Perplexity Search and its API platform. The stack comprises Ivy (Rust HTTP gateway), Tulip (gRPC scheduling and batching), and ROSE (Runtime-Optimized Serving Engine), which reuses LLM prefill and decode kernels rather than running a separate embedding engine. Optimizations include whole-model CUDA graphs with lazy capture and a LazyTensor abstraction that overlaps CPU batch preparation with in-flight GPU work. Benchmarks are reported against vLLM v0.22.0 in BF16, with FlashAttention 4 generally fastest but FlashInfer 3 winning on Qwen-based models at very long sequence lengths.
33
The OpenClaw 2.0 release moves your sessions into SQLite
OpenClaw 2.0 migrates AI agent sessions to SQLite, adds guided credential setup, and flags shared-session controls as not a security boundary.
OpenClaw 2.0, described as the largest update in the project's history, migrates sessions and transcripts into SQLite and adds guided setup that detects existing AI credentials from Codex, ChatGPT, Claude CLI sign-ins, API keys, and local Ollama or LM Studio models. The release expands multiplayer sharing while explicitly stating its permission controls are not tenant isolation or a security boundary, and that revoked access can briefly remain usable. Startup JavaScript requests fell from 140 to 45 and startup time from about 1.6 seconds to 575 milliseconds in simulated tests against a mocked Gateway. Automation wrappers must now inspect reported health because requesting --json does not waive risk acknowledgement.
35