PAIR-MED alternates policy and reward training for medical reasoning
Original title: PAIR-MED: Policy-reward Alternation for Iterative Refinement in Multilingual Medical Reasoning
8B and 9B models across 12 languages lift medical-subset accuracy by 3.01 to 7.47 percent over matching Qwen backbones.
The authors argue a fixed process reward model drifts from an evolving policy during reinforcement learning. PAIR-MED alternates updates: the policy generates multilingual trajectories that retrain a medical process reward model, which then scores the next policy update. After supervised initialization, medical step supervision keeps the reward model grounded while process, outcome, and language rewards guide the policy. PAIR-PRM and PAIR-MED are trained at 8B and 9B scales across 12 languages.
- Gains are reported on medical subsets of Global-MMLU and MMLU-ProX.
- PAIR-MED-9B raises a joint consistency and accuracy metric by 12.88 percent in-domain on CUREMED-BENCH.
- The same metric rises 7.45 percent for out-of-domain languages.
- Analyses cover PRM adaptation and the contribution of each training component.
From the original article
1 min read at arxiv.orgLarge language models demonstrate strong medical question-answering capabilities, yet reasoning performance remains uneven across languages. Process reward models (PRMs) provide step-level feedback for improving reasoning, but a fixed PRM may become increasingly misaligned with the trajectory distribution of an evolving policy during reinforcement learning, thereby reducing the reliability of its supervision.
By Fan Gao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.