StudentBench: AI and human tutoring yield equivalent GRE learning gains
StudentBench finds AI tutoring statistically matches expert human tutors on GRE learning gains at far lower cost.
StudentBench is a public AI teaching-evaluation suite drawing on more than 175,000 student-AI messages. Across 2,383 participants, AI tutoring was statistically equivalent to expert human tutoring on Quantitative and Verbal GRE questions (p = .015), and the best AI tutor surpassed the human tutor on average in five of seven domains. One tutor matched human learning gains (p = .044) at about 918 times lower cost, USD 0.0052 versus USD 4.81 per percentage point gained. Expert tutors also compared LLM lesson plans and practice problems in 2,028 pairwise rubric evaluations.
- Study measured GRE gains across 2,383 people and over 175,000 student-AI messages.
- AI tutoring was statistically equivalent to expert human tutoring (p = .015).
- Best AI tutor beat the human tutor on average in five of seven domains.
- One tutor matched human gains at about 918 times lower cost.
From the original article
1 min read at arxiv.orgArtificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities.
By Curtis Northcutt · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.