AI text humanizers drop or change checkable facts in 35-65% of docs
Original title: Humanized but Still True? Auditing Facts and Detection in AI-Text Humanizers
In an October 2026 leaderboard cycle, 10 of 11 tools still altered numbers or names in at least 20% of documents.
A study checks whether numbers, dates, names and quoted spans survive commercial AI “humanizer” rewrites. In the October 2026 cycle of a public leaderboard, all 11 tools changed or dropped at least one such unit in 35-65% of documents that contained one; the September cycle was similar (19-50%). An open 12B humanizer kept at least 97% of numbers and names in fact-dense U.S. federal text but reworded 67-78% of quoted spans. On matched federal texts, Jev separated human originals from humanized rewrites with AUROC 0.864, while perplexity detectors scored below chance on 2026 AI-styled rewrites.
Unconfirmed: The retention–evasion correlation is weak; the 95% CI is -0.65 to 0.08.
- Most changes were to quoted spans, not only numbers and names.
- Unit retention and detector evasion were not clearly related (Spearman rho = -0.46).
- Jev called 24% of human originals AI at a 0.5 threshold.
- Jev’s AUROC was 0.18-0.22 higher than Binoculars, Fast-DetectGPT and RADAR.
- The authors release the checker and per-item results.
- Countries
- United States
- AI models
- JevBinocularsFast-DetectGPTRADAR
From the original article
1 min read at huggingface.coAI "humanizers" rewrite machine-generated text so that detectors classify it as human-written. Public leaderboards score evasion and embedding similarity, but not whether checkable details survive. We check whether numbers, dates, person and organization names, and quoted spans survive each rewrite, using a rule-based checker validated by two AI annotators (precision 0.
By Ziyuan Wang, Genliang Zhu, Xiaobo Chen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.