Simulated students that make realistic mistakes help AI tutors learn faster

Training an AI tutor with a large, diverse group of students is "prohibitively expensive and time-consuming," the authors write in their paper. As a result, improvements to these tutors have lagged behind advances in AI models.

The researchers propose using digital replicas of students to provide quick feedback in their place. Their system, called StudentSim, builds a separate replica for each student, even when very few records of that person's work are available.

Student replicas need to make mistakes and learn from guidance

Existing approaches handle only one of two necessary skills, according to the researchers. Some models learn from real student data and reliably reproduce a student's behavior, but they can't use a tutor's explanations. Others are language models prompted to act as students. They readily follow the tutor's hints but fail to match the abilities of the student they're supposed to mimic.

StudentSim turns both skills into measurable goals. It measures how closely a replica matches a student's answers, including typical mistakes, and how readily it revises an answer after the tutor helps. Tutor training needs both a realistic starting point and a simulated student that responds to instruction.

Training in two stages makes limited student data usable

The researchers' biggest obstacle is a lack of data. In the English writing dataset, the median student has written just three essays, and more than two-thirds have written five or fewer. Training a replica directly on so few examples fails, the researchers say, because the model overfits to those examples.

StudentSim instead trains in two stages. First, a base model learns from the pooled data of all students in a subject. It learns common mistakes and how students revise their answers after a tutor's hint.

The researchers then tailor that model to an individual student using the few records available for that person. Across all subjects, the system uses Alibaba's Qwen3-4B-Instruct language model as its base.

StudentSim outperforms GPT-5.4 in chess, English, and math

The researchers tested the method on 60 students across chess, English as a foreign language, and math. They used public datasets containing records from real learners. StudentSim outperforms the larger GPT-5.4 language model in all three subjects when GPT-5.4 is prompted to act as a student. In chess, StudentSim correctly predicts a player's next move about twice as often and almost always follows corrective guidance. GPT-5.4 and specialized chess models fall behind.

Each existing method has a different weakness, the researchers say. GPT-5.4 follows hints but doesn't reproduce a particular student's mistakes. Chess models match a player's behavior but can't understand verbal hints and ignore them. In one position, three real players chose three different moves. StudentSim reproduced each player's choice, while a chess model predicted the same most likely move for all three. GPT-5.4 got all three wrong.

Training with a simulated student improves a chess tutor

As another proof of concept, the researchers used a student replica to improve a chess tutor. Professional chess players evaluated three versions, one without this training, one trained with GPT-5.4 as the student, and one trained with StudentSim.

The StudentSim-trained tutor scored highest on all three measures. It made the fewest serious factual errors and received the highest scores for explanation quality and adaptation to the individual student. In this case, the student preferred questions that guided them toward a solution rather than direct instructions.

The tutor trained with GPT-5.4 scored worse on factual accuracy than the tutor that received no extra training.

The researchers say this is only a proof of concept, not a claim to have built the best tutor. Chess works as a test case because an engine can objectively judge whether a move is good in any given position. Essay writing and open-ended math are harder because they lack reliable scoring functions for free-form answers.

Next, the team wants to model how students acquire, retain, and forget knowledge over many practice sessions. The code is available on GitHub.

Researchers used AI agents to replicate about 1,000 real people in 2024, based on two-hour interviews with each participant. A separate study showed how error-prone these replicas can be. Nine open language models tasked with mimicking user behavior on X, Bluesky, and Reddit became less accurate in their content as they sounded more human.

Microsoft is also testing AI tutors with real students. In a pilot project in Nigeria, students worked with Copilot twice a week for six weeks. Their test-score gains were equivalent to nearly two additional years of learning.

OpenAI and Google offer their own learning modes through Study Mode and Guided Learning. These rely on system instructions and models fine-tuned for teaching, but neither maintains a model of the individual learner. Without that adaptation, AI assistance can hurt performance. Studies show that users perform worse after brief AI assistance than people who worked on their own from the start.