TokenRouter: Efficient Serving System for Token-Level LLM Routing
TokenRouter raises token-level LLM routing decode throughput by 2.01–64.15× over existing serving systems.
1 stories, newest first · one card per story
TokenRouter raises token-level LLM routing decode throughput by 2.01–64.15× over existing serving systems.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views