onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction
onPanda corrects the first bad token and regenerates, cutting median alignment-annotation time by 52%.
onPanda annotates LLM alignment data and agent trajectories by token-level correction: the annotator fixes the first inappropriate token, the system truncates, and generation continues from the corrected prefix. A small controlled study reports a 52% cut in median annotation time versus manual post-editing, while most tokens remain model-generated and suitable for on-policy SFT and preference data. Recorded corrections also provide position-level positive-negative pairs. The authors release the Panda-CVL dataset and a token-level correction benchmark, and the tool can connect to external harnesses.
- Annotators correct the first bad token, then generation resumes from that prefix.
- Median annotation time fell 52% versus manual post-editing in a small study.
- Outputs stay mostly on-policy and supply paired positive-negative token corrections.
- Authors release Panda-CVL and a token-level correction benchmark.
From the original article
1 min read at huggingface.co
We present onPanda, an interactive tool for efficiently annotating LLM alignment data and agent trajectories. onPanda adopts token-level correction as its core interaction: while reading a model response, the annotator locates the first inappropriate token and either picks a substitute from the model's candidate tokens or types the correct text via free-form editing.
By Lei Yang, Mengyin Liu, Jia Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.