Pydantic schema prompts fail differently across nine LLM setups
Of 3,997 attempts, fenced JSON was the top failure, and only 48.0% of 681 fenced cases passed after fence removal.
Stories per day · 30 days
2 RSS
Relatedanthropic1clairic1cloudflare1document-extraction1llm-evaluation1openai1pydantic1schema-validation1workers-ai1
4 stories, newest first · one card per story
Stories tagged #structured-output
Cloudflare previews Clairic extraction model that skips token generation
In one claims workflow, a Clairic pipeline finished in 19.3 seconds and cost 49% less than an all-LLM version.
1
Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It
Swapping option names from 0/1 to no/yes reverses Jev decisions, dropping AUC from 0.94 to 0.23.
harshatheg/Qwen-2.5-1B-RLCD — new model trending #30 on Hugging Face
A community MLX inference engine evaluates constrained JSON schema fields in parallel on Apple Silicon, reporting 5.6-7.0x latency speedups with guaranteed schema validity.
6