onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction

onPanda cuts median alignment-annotation time by 52% by having annotators correct the first bad token and regenerate from the corrected prefix, keeping data on-policy for SFT and preference training.

onPanda is an interactive annotation tool for LLM alignment data and agent trajectories, listed on Hugging Face daily papers on 2026-09-20 and on arXiv (cs.AI, cs.LG, cs.CL) on 2026-09-21 under the same title. The annotator locates the first inappropriate token in a model response, corrects it, and the system truncates the remainder so generation continues from the corrected prefix; the arXiv report adds that this locate-correct-continue loop repeats until the response is satisfactory, a detail not spelled out in the Hugging Face listing. A small controlled study reports a 52% reduction in median annotation time versus manual post-editing. Because most tokens still come from the model itself, outputs preserve the sampling distribution and remain largely on-policy, suitable for supervised fine-tuning and preference data, while recorded corrections supply fine-grained supervision with naturally paired positive-negative token samples. The authors release the Panda-CVL annotated dataset and a token-level correction benchmark, and the Hugging Face report notes the tool can connect to external harnesses. The two reports agree on all figures and claims.

• onPanda appeared on Hugging Face daily papers on 2026-09-20 and on arXiv (cs.AI, cs.LG, cs.CL) on 2026-09-21 under the same title.
• Annotation works by token-level correction: the annotator fixes the first inappropriate token, the system truncates, and generation continues from the corrected prefix.
• Only the arXiv report describes the loop repeating until the response is satisfactory; the reports do not otherwise disagree.
• A small controlled study reports a 52% reduction in median annotation time versus manual post-editing.
• Most tokens remain model-generated, keeping data largely on-policy for SFT and preference training.
• Corrections yield fine-grained supervision with naturally paired positive-negative token samples.
• Authors release the Panda-CVL annotated dataset and a token-level correction benchmark.
• The tool can connect to external harnesses (per the Hugging Face report only).