Direct Nash Optimization aligns language models with general preferences
DNO-Prct iterates on-policy sampling, rating, and contrastive updates to raise length-controlled win rates on AlpacaEval 2.0 for Llama 3.1 8B and Mistral 7B Instruct models.
Stories per day · 30 days
5 stories, newest first · one card per story
Stories tagged #preference-alignment
Collaborative Personalized Preference Alignment for LLMs under Data Deficiency
Approximate Pareto Optimality learns shared aligner initializations for few-shot LLM personalization from 20 examples.
ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker
ZooWork-ShopRanker releases 0.6B, 4B, and 8B e-commerce rerankers trained on LLM shopping preferences.
A Zeroth-Order Paradigm for LLM Preference Alignment
Researchers propose ComPO, a zeroth-order comparison-based preference alignment method with convergence guarantees that mitigates likelihood displacement in LLMs.
A Zeroth-Order Paradigm for LLM Preference Alignment
ComPO is a zeroth-order preference alignment method using comparison oracles to mitigate likelihood displacement across Mistral, Llama, Gemma, and Qwen3 models.