Program-Verified Self-Evolution for Vision-Language Models
VQS generates program-verified vision questions, labeling 94% correctly and lifting Qwen3-VL by up to 3.84 points.
Self-evolving vision-language models often label generated questions by majority vote or a model judge, which human evaluation found wrong 24% and 18% of the time. VQS parses each image into a structured record and uses fixed programs to write the question and compute its answer, while the model checks only short claims. Human raters found 94% of VQS answers correct, versus 76% for majority voting. Across ten benchmarks, VQS improves Qwen3-VL by up to 3.18 points at 2B, 4B, and 8B, reaching 3.84 points at 2B after three rounds.
- Majority-vote labels were wrong 24% of the time; judge labels 18%.
- Programs write questions and answers from parsed scene, chart, or diagram records.
- Human raters marked 94% of VQS answers correct versus 76% for voting.
- Qwen3-VL improves by up to 3.18 points at 2B, 4B, and 8B.
- Gains reach 3.84 points at 2B after three training rounds.
From the original article
1 min read at huggingface.co
Self-evolving vision-language models train on questions they generate from unlabeled images. Since these questions have no gold answers, prior methods label them by majority vote over sampled answers or by a model judge. In a human evaluation, we find that 24\% of majority-vote labels and 18\% of model-judge labels produced during self-evolution are wrong.
By Ahmed Heakl, Sungik Choi, Moontae Lee · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.