TUPLE-Bench finds typed decision outputs leak withheld causes
Original title: TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models
Across nine models, 91.88% to 99.93% of responses revealed a withheld cause through selection or scores.
Researchers introduce TUPLE-Bench to measure privacy leakage from typed decision models that return categories, probabilities, and sometimes confidence. Instructions to withhold a fine-grained cause failed across nine models. Full outputs still enabled 62.85%–75.35% recovery when causes were excluded from candidates, versus 51.04%–57.64% from the selected category alone. They propose workflow-constrained privacy quantization to cut recovery while preserving specified downstream cost.
- Cause recovery stayed at 91.88%–99.93% despite withhold instructions.
- Full outputs beat category-only recovery, which was near a 50% chance.
- Full outputs lowered mean downstream cost in all 27 comparisons.
- Confidence fields increased leakage without consistently reducing cost.
- WCPQ reduced secret recovery while keeping specified workflow behavior.
From the original article
1 min read at arxiv.orgTyped decision models often return more than a selected category. They also expose probabilities for each candidate and, sometimes, a separate confidence score. These fields support downstream decisions but may reveal information beyond a recipient's authorization.
By Yiqu Yang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.