vLLM constrained decoding lifts Kimi tool-call schema accuracy to 100%
Original title: Fixing Tool-Call Accuracy in vLLM
Serving Kimi K2.6 on vLLM 0.28 matched the expected schema only 92% of the time, the authors report.
Ubicloud engineers say unconstrained tool calls from open-weight models often break agent workflows because malformed calls pollute context and force retries. With vLLM 0.28, Kimi K2.6 produced the expected tool-call schema only 92% of the time. vLLM 0.31 exposes constrained decoding as a server-side flag that, they say, reaches 100% schema accuracy on newer Kimi and DeepSeek models. The post explains schemas, Moonshot’s K2VV ToolCall benchmark, and trigger similarity versus schema accuracy.
Unconfirmed: The 92% and 100% figures are the authors’ serving measurements, not an independent benchmark result.
- The authors equate 92% accuracy with about 2 failed calls in every 25 attempts.
- K2VV ToolCall uses 2000 requests for trigger similarity and schema accuracy.
- Constrained decoding is described as guaranteeing schema adherence at serving time.
- OpenCode is used as the example harness sending a webfetch tool schema to Kimi.
- Organizations
- UbicloudMoonshot AI
- AI models
- Kimi K2.6
From the original article
8 min read at ubicloud.com
When I use an open-weight model, I see results that are nowhere close to what the quality benchmarks indicate. Tool call accuracy is an important part of serving open-weight models. When your tool calls aren’t accurate, the model builds up on nonsensical context.
By coffeemug · ubicloud.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.