Meta's Cap-and-Track steers LLM inference using capacity refusals
In a production A/B, a classifier ran on 43% fewer H100s than a queued fixed-rate shaper.
1 stories, newest first · one card per story
In a production A/B, a classifier ran on 43% fewer H100s than a queued fixed-rate shaper.
Today 1.2K readers · 1.3K views · 132 reads · all time 215K views