42
55
60
30
30
60
60
55
60
55
30
60
60
60
60
60
60
60
60
60
30
60
57
55
1Password's AI patching benchmark is misleading
Trail of Bits reanalysis says 1Password's 26% AI clean-fix rate is misleading; 86% of eligible patches blocked exploits.
Trail of Bits critiques 1Password's FLAWED AI patching benchmark, arguing its 26% clean-fix headline mixes trials where agents were instructed to apply wrong fixes (22% of data) with trials that prohibited compiling or testing (36%). Restricting to reasonable conditions, 2,634 of 3,067 patches (86%) blocked the supplied exploit. Trail of Bits also reports 12.5% of 2,265 developer first fixes failed in its own 2024-2026 assessments, and released post-patch-validation and review-walkthrough agent skills.
45
55
47
60
42
47
60
60
60
30
30
30
30
30
30
30