Training Witnesses: Trusting the Training without Trusting the Trainer
Witnesses lets trainers certify language-model runs so others can audit training without full reproduction.
Witnesses is a method for certifying neural-network training, data usage, and evaluation so reviewers do not have to reproduce expensive runs. It relies on fast behavioral fingerprints with occasional replay challenges, supports exact data inclusion and exclusion queries, and rejects bad runs with amplifiable probability at low trainer overhead. The authors tested language-model training from 100 million to 2 billion parameters under DDP and FSDP. They also propose a self-regulating leaderboard of auto-certified runs to share baselines and improve reproducibility.
- Witnesses certifies training, data use, and evaluation without full reruns.
- Fingerprints plus replay challenges audited models from 100M to 2B parameters.
- The method covers DDP and FSDP with minimal overhead for the trainer.
- Authors propose an auto-certified leaderboard for shared reproducible baselines.
From the original article
1 min read at arxiv.orgProgress in machine learning cannot outpace our ability to verify it. With an explosion in papers today, every scientific claim rests initially on trust in the trainer, leading to uneven evaluation, baselines, and forestalling of reliable progress. Traditionally, the burden of verification falls on the reader, who must reproduce expensive training runs.
By Houjun Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.