MPO aligns LLM agent credit assignment at the turn level
The method assigns credit at assistant-turn granularity while keeping finer token-level allocation, outperforming RL baselines on agentic tasks.
Stories per day · 30 days
2 RSS
Relatedagent-training1ai-agents1alignment1credit-assignment1cve-2026-656161differential-privacy1exploitgym1llm-agents1openai1preference-feedback1privilege-escalation1reinforcement-learning1reward-modeling1zero-trust1
19 stories, newest first · one card per story
Stories tagged #rlhf
Paper bounds private sparse reward estimation from preference feedback
Sample-DP and Label-DP rates depend on sparse features s* rather than full model size d.
From Training to Escalation: Forensic Analysis and Local Recreation of RL/RLHF-Induced Privilege Escalation in AI Agents (CVE-2026-65616)
Paper recreates an AI-agent privilege-escalation path and the CVE-2026-65616 cryptographic bypass from OpenAI ExploitGym conditions.
A Scoping Review and Experimental Study on Reinforcement Learning from Human Feedback for Human-Robot Collaboration
Review of 20 RLHF human-robot papers finds user-initiated feedback better captures perceived navigation safety.
EasyPPO: Stabilizing the Critic Is Key
EasyPPO stabilizes the PPO critic for LLM training and outperforms vanilla PPO on coding, math, and search.
Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling
P-TTT personalizes reward models with test-time fast weights, beating in-context preference conditioning without backprop.
Diffusion Reward Models
DRM models rewards with a diffusion transformer, capturing multimodal preferences and improving RLHF decisions.
Rufus-Air: An Open LLM Post-Training Recipe
Rufus-Air releases a reproducible eight-stage post-training recipe on GLM-4.5-Air-Base that beats the official post-trained release.
Jev: System One models for Prod, not God — with Diogo Almeida, CEO, TypeSafe AI
TypeSafe CEO Diogo Almeida explains Jev, a System One model trained with RLCD for calibrated production decisions.
A new kind of AI model from a ChatGPT inventor is thrilling developers
TypeSafe AI, founded by ex-OpenAI RLHF inventor Diogo Almeida, released Jev, a non-LLM transformer outputting calibrated probabilities for cheap software automation.
2
A Zeroth-Order Paradigm for LLM Preference Alignment
Researchers propose ComPO, a zeroth-order comparison-based preference alignment method with convergence guarantees that mitigates likelihood displacement in LLMs.
Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flattening
Researchers identify Value Flattening in PPO critics for LLM RL and propose SP^3O sparse value supervision, improving Qwen3-Base training.
A Zeroth-Order Paradigm for LLM Preference Alignment
ComPO is a zeroth-order preference alignment method using comparison oracles to mitigate likelihood displacement across Mistral, Llama, Gemma, and Qwen3 models.
Inside ‘Project Lily’: The Humans Reading Your ChatGPT Chats
404 Media reveals OpenAI's 'Project Lily' has hundreds of contractors reading real ChatGPT user prompts, exposing sensitive personal data despite privacy filters.
OpenAI adds a prominent AI doomer to its board of directors
OpenAI appointed alignment researcher Paul Christiano to its Foundation board's Safety and Security Committee amid scrutiny over AI agent escape incidents.
PLC-DPO: Posterior Label Correction in Noisy and Ambiguous Preference Optimization
PLC-DPO corrects noisy preference labels during DPO by routing each pair as clean, flip, or tie using calibrated policy-reference margins.
AgenticGen: Reward-Guided Agentic Video Generation for Advertising
AgenticGen applies DPO and GRPO reward-guided reasoning to ad video generation, improving TikTok CTR 2.72%, CVR 2.63%, and Advv 9.61%.
I wrote an AI textbook — how long until AI can do it better?
AI researcher Nathan Lambert argues LLMs remain weak at long-form technical writing, questioning whether models can autonomously organize scientific knowledge for breakthroughs.
5 useful things you'll learn in my new post-training textbook (shipping now!)
Nathan Lambert's new RLHF and post-training LLM textbook covers PPO, GRPO, GSPO, CISPO and related techniques, freely available online.