Leaker says Apple Harbor will run local AI models on Macs
Harbor, in a Claris private preview, pools Thunderbolt 5 Macs so open-weight models stay on device.
Stories per day · 30 days
2 RSS
Relateddram2gguf2memory-supply2openai2privacy2quantization2agentic-coding1apple1apple-silicon1browser1budgeting1claris1custom-chips1data-centers1flint1gemma1github1harbor1
22 stories, newest first · one card per story
Stories tagged #local-inference
mradermacher publishes GGUF quants of gemma-4-31B-it-ARA-heretic
Static GGUF files from Q2_K through Q8_0, plus mmproj supplements, are listed as the repo trends at #30.
Sam Altman says DRAM capacity should go to data centers, not PCs
In a Cheeky Pint interview, he called home DRAM and local inference quite terrible because memory sits idle most of the time.
Altman says local DRAM use wastes scarce memory supply
He argues cloud utilization is better than local inference and says OpenAI is designing a custom AI chip called Jalapeno.
Opera previews WebAI standard for running models in the browser
The early preview in Opera One can use models such as Gemma or Apple’s on-device model, with processing kept on the device.
Guide covers running Underdog Saluki 27B for tool calling
The title describes Saluki 27B as a 2-bit Qwen3.8-27B build; the page body was not captured.
1
Recursive releases Flint, an open MoE coding model built by Eureka Machine
The 40B-parameter model keeps 2.5B active per token, ships under Apache 2.0, and its benchmark lead is unverified.
1
Llama.cpp on a jailbroken PS5: run any GGUF model natively on the console
A GitHub project ports llama.cpp to a jailbroken PlayStation 5 to run GGUF models natively.
Show HN: Pulse Local, a Mac budgeting app whose AI models run on your machine
Pulse Local is a Mac budgeting app that runs its AI models entirely on device.
2
Show HN: Offrun – manage every coding agent from one workspace
Offrun is a free Mac app that manages multiple coding-agent CLIs in one local workspace.
3
From the creator of Redis; run LLM locally with ds4
Antirez's ds4 inference engine runs DeepSeek V4, GLM 5, and Qwen3.8 locally on high-memory Mac, CUDA, and ROCm machines.
9
Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia
Janus is a Go binary that runs GGUF models on AMD, Intel, and Nvidia GPUs via Vulkan.
Ollaya – Ollama for open-source, Jev-style decision models
Ollaya is an open-source local runtime for fast Jev-style decision models, compatible with TypeSafe.
8
Laya (OS Jev) on Mac M4 CoreML Offline (45 decisions per second)
A gist shows the Laya (OS Jev) model running fully offline on an Apple M4 Mac via CoreML at 45 decisions per second.
Best Open-Source Agent Harnesses for Local LLMs in 2026
A 2026 guide ranks 11 open-source agent harnesses, including OpenCode, Goose, Cline, and OpenHands, by local-LLM support, licensing, and safety controls.
7
Long-Lived Characters, Local Inference: Incremental Memory Maintenance for Game NPCs
Researchers present incremental KV-cache memory maintenance for long-lived game NPCs running locally on a quantized Qwen hybrid model.
JustFit: 200K-Token LLM Serving on a 24 GiB Laptop with Just-in-Time State Management
JustFit MLX runtime serves 200K-token contexts for Qwen3.8-27B on a 24 GiB MacBook via just-in-time state management.
Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed from four SSDs
A GitHub project streams Moonshot's 2.8T-parameter Kimi K3 from four SSDs to run at 1 token/s on a MacBook Pro.
Nous Research Adds One-Click Local Model Setup to Hermes Desktop
Nous Research's Hermes Desktop now offers one-click local model setup that reads hardware, picks a fitting quantization, downloads weights, and configures llama.cpp automatically.
16
Microsoft’s Project Zenith is a ‘distraction-free Windows experience’ for developers
Microsoft branded its developer-optimized Windows setup Project Zenith, with 64GB+ memory devices from AMD that run 30B+ parameter AI models locally.
2
Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI
Hugging Face released @huggingface/kernels, a library offering 200+ WebGPU compute kernels to accelerate AI inference locally in browsers.
Meta is back with Muse Glimmer: local, agentic, multimodal, and open source
Meta releases Muse Glimmer, an open-source model built for local, agentic, multimodal use.