Meta releases Muse Ember, a lightweight model for agent work
Meta prices Ember at 10 cents per million input tokens and pairs it with Muse Wonder for volume.
1
Stories per day · 30 days
2 RSS
Relatedbenchmarks2agent-evaluation1agents1coding-agents1early-stopping1llm-agents1meta1model-naming1model-scaling1monitors1muse-ember1reasoning-effort1search-agents1small-language-models1swe-bench1test-time-compute1tokenization1value-of-search1
14 stories, newest first · one card per story
Stories tagged #inference-cost
Ted Sanders says model size explains most LLM performance gaps
Bigger models generally cost more per token and recall obscure facts better, even when skill rankings differ.
SearchHorizonBench tests whether AI agents can value another search
On 1,632 held-out questions, historical calibration cut Brier error 16.2%, but point estimates explain under 2% of search-gain variance.
Higher LLM reasoning effort can cost more without better results
On easy tasks every effort level solves, the highest setting costs up to 42 times the lowest, and escalating after failure is often cheaper.
Specializing SLM vocabularies can cut inference cost by 77%
A Hugging Face post claims vocabulary specialization cut small-model inference cost by 77% in real applications; no further details were provided.
StopLoss aborts likely-failing coding-agent runs early
On SWE-bench Verified it cut cost per retained solve by 4.5 percent while keeping 99 percent of successful runs.
EdgeWise: Bringing Long-Horizon Agents to the Edge via Selective Cloud Delegation
EdgeWise delegates only key agent steps to the cloud, cutting inference cost by up to 62%.
Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?
BOTTLED benchmarks whether LLM agents can convert large repetitive workloads into cheaper reusable solutions.
Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routing
SaveRouter trains LLM routers on 33–41% of feedback and shortens cost payback by up to 9.5 times.
FragToken: Amplifying LLM Inference Costs through Noncanonical Token Generation
FragToken trains LLMs to emit longer non-canonical tokens, roughly doubling inference cost while keeping answers usable.
The Price of Intelligence Is Falling Rapidly
Epoch AI finds a given AI performance level became about 47% cheaper per quarter, roughly 13-fold yearly.
CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
CliffCompaction cuts long-horizon coding-agent context cost by up to 50% without rewriting retained text.
Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents
Growing Harness converts recurring agent control into reusable code, cutting LLM calls by 76 to 92 percent.
JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
A decision-only JEV judge nearly matches a top LLM judge at a fraction of the cost.