SCOPE compiles natural-language agent policies into predicate programs
Original title: SCOPE: Compiling Natural-Language Policies into Neurosymbolic Predicate Programs
Programs trained with a weak executor transfer to other models and match frontier accuracy at lower cost with more stable verdicts.
SCOPE compiles application-specific policies for LLM agents into executable programs of neural predicates, deterministic extractors, and composing code. Training jointly optimizes the program and its data, using adversarial counterexamples where the program diverges from the policy. On a benchmark of real-world policy tasks, the authors report the highest average accuracy versus prompt optimization, task decomposition, and workflow generation, plus more stable and interpretable verdicts than monolithic judges.
Unconfirmed: Accuracy and stability claims are the authors' benchmark results; the source does not name the competing systems or score tables.
- Policies cover multi-turn transcripts and tool-call trajectories, not only single-turn moderation.
- Programs optimized with the weakest executor transfer without further optimization.
- The paper says production deployment on inexpensive models can match frontier reasoning models.
- An ablation study is used to check the method's main components.
From the original article
1 min read at huggingface.co
Policy enforcement for LLM agents has shifted from single-turn content moderation to compound, application-specific policies evaluated over multi-turn transcripts and tool-call trajectories. Fine-tuned guardrail models are efficient but degrade on compound policies, and while LLM-as-a-judge approaches achieve higher accuracy, they incur high inference cost and tend to produce unstable predictions.
By Arnon Mazza, Elad Levi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.