StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?
Researchers introduce StudyBench, a physics benchmark showing self-evolution gains on textbook problems rarely transfer to olympiad-level questions.
StudyBench is a controlled physics benchmark splitting test data into an Application Set of difficult textbook problems and a Transfer Set of olympiad-level problems. Across three base models, representative self-evolution methods improved on the Application Set but rarely transferred to the harder Transfer Set. A guidance ablation reveals a Guidance Gap, and every method hits a Compute Plateau, indicating the remaining limits are method problems rather than data or compute problems.
- Measures how efficiently self-evolution converts raw material into transferable problem-solving capability.
- Textbook-problem gains rarely transfer to olympiad-level questions across three base models.
- Guidance Gap: best method closes only a fraction of what in-context guidance unlocks.
- All methods hit a Compute Plateau, so limits are method problems, not data or compute.
- Products
- StudyBench
- Organizations
- Tsinghua University
From the original article
1 min read at huggingface.co
Humans need to study only a handful of well-written textbooks to master a discipline and attempt its hardest problems. We argue that an ideal self-evolution method should share the same property, that is autonomously learning from raw training material for transferable problem-solving capability. However, we still lack a direct measurement for it.
By Yinghao Chen, Zixi Chen, Bingxiang He · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.