30
30
Speculative Decoding in vLLM on AMD GPUs
vLLM benchmarks speculative decoding on AMD Instinct MI300X and MI355X GPUs across five drafting methods including EAGLE-3 and native MTP.
The vLLM project documents draft-and-verify speculative decoding support for AMD GPUs via ROCm, comparing native MTP, Gemma 4 MTP, EAGLE-3, DFlash, and DSpark drafting approaches. Output-token throughput effects varied with drafting method, proposal length, model family, draft checkpoint, workload, and acceptance behavior. The post also covers how to enable each method plus practical tuning and observability considerations.
40
30
30
Give Your Coding Agents a Memory You Own
Hugging Face introduces Funes, a tool that gives coding agents persistent, self-owned memory outside vendor clouds.
A Hugging Face blog post presents Funes, an approach for giving coding agents a persistent memory that developers own and control. The piece targets agent workflows where context must survive across sessions without ceding data to third-party services. No article body was available in the feed, so specifics beyond the title are limited.
25
30
30
30
30
30
30
30
30
30
30
30
30
Traefik Labs brings independent verification to AI agent governance
Traefik Labs announces Sovereign Trust Plane in Traefik Hub, adding verifiable delegation, policy enforcement, and tamper-evident audit records for AI agent traffic.
Traefik Labs announced the Sovereign Trust Plane for Traefik Hub, generally available by September 30, 2026, providing delegated access, policy enforcement, and tamper-evident records for AI agent, tool, and API traffic. It implements the IETF ID-JAG draft with Okta Cross App Access and Janssen, enforces decisions through OpenID AuthZEN with OpenFGA and Cerbos, and commits cryptographic log fingerprints to transparency checkpoints verified by independently administered witnesses. The gateway also extends enforcement to MCP tool calls and the MCP server's backend API connection.
38
Rapidly scaling online storage to serve over 1 billion ChatGPT users
OpenAI's Habitat online storage platform now handles over 70 million requests per second and 500 PB of data for 1 billion users.
OpenAI details the evolution of Habitat, its online storage platform backing ChatGPT and other products, which began in mid-2024 as a Python client-side library over Azure Cosmos DB. Habitat now processes more than 70 million requests per second, serves over 500 petabytes of data across nearly 40 geographic regions, and supports over 1 billion users weekly. By mid-2025 the client library approach became brittle, so OpenAI moved Habitat into a standalone service to centralize deployments, observability, and multi-tenancy reliability. This is part one of a two-part series; a future post will cover read optimization and scaling the Azure Cosmos DB partnership.
30
Meet Redis LangCache: A Managed Semantic Cache That Cuts LLM API Costs by Up to 90% and Returns Cache Hits Up to 15x Faster
Redis launches LangCache, a managed semantic cache matching LLM prompts by meaning, cutting API costs up to 90% and returning hits up to 15x faster.
Redis LangCache is a fully managed semantic caching service in public preview on Redis Cloud, accessed via a REST API with Python and JavaScript SDKs. It embeds incoming prompts, vector-searches stored entries, and returns a cached response when similarity clears a configured threshold, skipping the LLM call entirely. Redis claims up to 90% cost savings and up to 15x faster cache hits; a demo run showed 0.37 seconds versus 2.232 seconds direct inference (about 6x) with zero LLM tokens. Customer Mangoes.ai reports a 70% hit rate, 70% lower LLM spend, and 4x faster responses on a patient-care voice app.
30
The VMs Powering Mobile Agents (Instinct, Claude Code)
A teardown reveals Claude Code runs in Firecracker microVMs with a Rust PID 1 and MITM'd egress, while Instinct rents E2B sandboxes with git-based memory.
The author inspects the virtual machines hosting cloud agents: Claude Code runs in a Firecracker microVM with a custom Rust init (process_api) as PID 1, a 324 MB Bun harness on a read-only disk, and 443-only MITM'd SSE egress to api.anthropic.com with host-rotated OAuth tokens and no inbound access. Instinct rents E2B sandbox-as-a-service Firecracker microVMs (Ubuntu 22.04, 2 vCPU, 1.9 GB RAM) where agent memory is a git repo of Markdown committed by the agent and pushed to S3 as a single bundle, using short-lived STS credentials. Both platforms rely on Firecracker, differing mainly in fleet operator and guest boot configuration.
35
ToolHive: The open-source way to run any MCP server securely
Stacklok's open-source ToolHive runs Model Context Protocol servers in isolated containers with per-request identity enforcement, audit logging and a signed registry.
ToolHive, shipped under Apache 2.0, containersizes MCP servers locally via Docker or Podman or in clusters through a Kubernetes operator, applying permissions, network filtering, and secrets management. The platform includes a Registry Server implementing the official MCP Registry API with signing and provenance verification, a Virtual MCP Server gateway with OIDC/OAuth single sign-on and OpenTelemetry traces, and a desktop Portal for one-click installs. The browser-based cloud UI is retired, so the desktop app and CLI are the supported rollout paths.
45
GitHub Introduces Project HydraFusion: Runtime Multi-Model Orchestration That Builds a Workflow Per Coding Task in Copilot CLI
GitHub's Project HydraFusion research preview builds per-task multi-model workflows (Single, Cascade, Critique) in Copilot CLI, reporting +4.9 quality at 67% lower cost on TerminalBench 2.1.
Project HydraFusion is a research preview available on all GitHub Copilot plans inside Copilot CLI that treats model routing as workflow selection, choosing among Single, Cascade (draft plus quality gate), and Critique (cross-family reviewer) execution patterns per request. Against Claude Opus 5 baselines at medium reasoning, fixed HydraFusion policies cut estimated cost 67% while adding 4.9 quality points on TerminalBench 2.1, and cut cost 36% and 65% with slight quality dips on DeepSWE and CheckpointBench. Billing is per token at each underlying model's standard rate; there are no open weights or self-hosting options.
42
Halo-record: Open-source audit trails for AI agents
Developer Brian Kuan released halo-record, an open-source Python package creating tamper-evident, hash-chained audit logs of AI agent actions.
Halo-record is a roughly 5,300-line Python package with no runtime dependencies that records agent tool calls, model calls, data access and approvals into an append-only, hash-chained log that customers can verify without vendor trust. Adapters ingest records from OpenTelemetry spans, LangChain, MCP servers and gateway logs, with secret and PII values auto-redacted. The author plans to fund the work through a hosted witness service that stores the record count and head hash to prove completeness, citing mandates like AIUC-1, the EU AI Act, and insurers. The article cites the July Hugging Face intrusion, where an autonomous agent took roughly 17,600 actions over five days and manual reconstruction of its activity was impractical.
35
The Evolution of the Agent Harness
Latent Space essay argues late-2025 agent gains came from models and harnesses maturing together, with harness logic absorbed into model weights.
The piece defines the agent harness as everything beyond model weights—tools, context, memory, guardrails—and charts its evolution from ReAct prompting (October 2022) through AutoGPT's premature autonomy, Cursor/Copilot's human-in-the-loop retreat, and Devin's roughly 15% success rate, to o1's capability overhang and Claude Code's February 2025 terminal agent with permission rules. It argues the Christmas 2025 jump cited by Transformer co-inventor Lukasz Kaiser reflected model and harness curves crossing, and that remaining harnesses will serve human attention rather than the model.
32
GitLab 19.3 helps enterprises scale agentic development securely
GitLab 19.3 runs its Duo Agent Platform AI Gateway inside Dedicated single-tenant environments and adds Secrets Manager plus agentic SAST remediation.
GitLab 19.3 lets GitLab Dedicated customers run the Duo Agent Platform AI Gateway within the same single-tenant environment and region, with support for bring-your-own inference models. The release adds Secrets Manager in limited availability scoping secrets to environment and branch across Kubernetes, Terraform, and OpenTofu, plus bulk SAST false positive detection and agentic vulnerability resolution generating ready-to-merge fixes. Flow Creator Agent creates automation flows from plain-language descriptions, and GitLab Credits usage caps are now generally available.
32
30
30
30
30
30
30
30
30
30
30
30
30
30