Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation
Study shows specialists trained on question-answer pairs implicitly select latent reasoning trajectories, and tuning choices control the precision-generalization trade-off in distillation.
The work demonstrates that specialist optimization implicitly selects from a latent trajectory space when specialists are trained only on question-answer pairs without explicit reasoning supervision. Using student distillation as an agnostic probe across 27 specialist-student pairings, specialization-generalization profiles correlate exceptionally strongly. Explicitly controlling the specialist's distributional drift systematically shifts both teacher and distilled student along a controllable trade-off between domain precision and general-capability retention across chemistry, physics, and multilingual settings, even across divergent model families.