vLLM constrained decoding lifts Kimi tool-call schema accuracy to 100%
Serving Kimi K2.6 on vLLM 0.28 matched the expected schema only 92% of the time, the authors report.
2
Stories per day · 30 days
2 RSS
Relatedablations1deepseek1inference1kimi1llm-reasoning1open-source-models1output-formats1tool-calling1vllm1
4 stories, newest first · one card per story
Stories tagged #constrained-decoding
Constrained decoding effects depend on the constraint type
Across eleven open-source models, apparent losses mostly came from mismatched prompts and formats rather than decoder masking.
Structural Automata for Neural Semantic Parsing
Accepting-path decoding guarantees valid semantic parses and lifts Smatch++ under tight output-length limits.
harshatheg/Qwen-2.5-1B-RLCD — new model trending #30 on Hugging Face
A community MLX inference engine evaluates constrained JSON schema fields in parallel on Apple Silicon, reporting 5.6-7.0x latency speedups with guaranteed schema validity.
6