SpecPilot raises speculative decoding speedup from 4.46x to 10.87x
SpecPilot is an agentic loop that discovers algorithms and optimizations for speculative decoding across deployments.
Stories per day · 30 days
2 RSS
2 stories, newest first · one card per story
Stories tagged #systems-optimization
Φ-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?
Researchers release Phi-Bench, a benchmark evaluating frontier LLMs on open-ended, long-horizon engineering and optimization of the LLM infrastructure stack.