Skip to content
NewA calmer, faster ZeroHour

Bait probes rewards to limit harmful agentic reinforcement learning

Original title: Bait: Mitigating Harmful Agentic Reinforcement Learning via Probing Reward Function

arXiv cs.CRby Tiansheng Huang ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

Authors say a roughly $20 API attack can fully undo alignment of a model such as Inkling.

The paper reports that harmful agentic reinforcement learning through a service API can strip safety alignment from a large language model. The authors say a degraded model could then assist real-world harm, including exploiting software flaws and code-reuse attacks. Their Bait defense probes the reward function so harmful rewards are not learned while benign agentic rewards still are, and they argue an adaptive attacker who knows Bait cannot fully defeat it.

Unconfirmed: The $20 cost, complete alignment loss, and downstream misuse are the authors' experimental claims.

Key facts
  • The claimed attack cost is as low as $20 through rounds of harmful agentic RL.
  • Inkling is the example model whose alignment they say can be fully subverted.
  • Bait still learns from a benign reward that promotes benign agentic tasks.
  • Authors say knowledge of Bait does not let an adaptive attacker perfectly subvert it.
  • Source code is published at github.com/git-disl/Bait.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

We expose a safety vulnerability on nascent agentic reinforcement-learning-as-a-service. Specifically, with a cost as low as \$20, we show that safety alignment of a large language model (e. g.

arXiv cs.CR

By Tiansheng Huang · arxiv.org

Read the full article at arXiv cs.CR

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.