A Zeroth-Order Paradigm for LLM Preference Alignment
Researchers propose ComPO, a zeroth-order comparison-based preference alignment method with convergence guarantees that mitigates likelihood displacement in LLMs.
32
Researchers propose ComPO, a zeroth-order comparison-based preference alignment method with convergence guarantees that mitigates likelihood displacement in LLMs.
ComPO is a zeroth-order preference alignment method using comparison oracles to mitigate likelihood displacement across Mistral, Llama, Gemma, and Qwen3 models.
Today 301 readers · 316 views · 139 reads · all time 167K views