Piloting the world's first double-blind AI evaluations

Google DeepMind is piloting the world's first double-blind AI evaluations, a new methodology intended to improve evaluation integrity and reduce bias.

Google DeepMind announced a pilot of double-blind AI model evaluations, described as the first of its kind. The approach is designed to reduce contamination and bias in model assessments by keeping evaluators and model identities hidden from one another. Details on participating models and protocols were not provided in the announcement text.

• DeepMind pilots double-blind AI evaluation methodology
• Aims to reduce bias and contamination in benchmarks
• Positioned as a first-of-its-kind evaluation practice