Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It
Swapping option names from 0/1 to no/yes reverses Jev decisions, dropping AUC from 0.94 to 0.23.
The paper finds that schema-constrained decision models can return valid options while following the semantic polarity of option names rather than the rubric bound to each name. On 1,200 workflow decisions, renaming the two options from 0/1 to no/yes changed 70.4 more answers per hundred and shifted AUC from 0.94 to 0.23. Across four predicates the effect was at least 7.4 times larger than a neutral-name control, while the type-error rate remained 0%. Random character-string names restored the neutral regime, and a mean-pooling readout flipped 4.1 times less often.
- Renaming 0/1 options to no/yes changes 70.4 more answers per hundred.
- AUC falls from 0.94 to 0.23, reversing the decision ranking.
- The effect is at least 7.4 times the neutral-name control.
- Type-error rate stays 0% even when decision accuracy collapses.
From the original article
1 min read at arxiv.orgTyped decision models are built for settings where model outputs are consumed directly by software. Instead of generating free-form text, they return a decision over a predefined set of options. By construction, every output conforms to the required schema.
By Yu Sun · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.