MotionGraphicsBench scores SVG motion edits with verification programs
Pass@1 across 47 configurations ranged from 6.0% to 67.7%, and verifiers agreed with humans 93.3% of the time.
Stories per day · 30 days
7 RSS
Relatedbenchmark2agent-evaluation1agent-harnesses1agentproofbench1ai-coding1claude1code-review1codex1coding-agents1computer-using-agents1deepseek1execution-traces1faros1github-copilot1lottie1motion-graphics1reinforcement-learning1science1
43 stories, newest first · one card per story
Stories tagged #verification
Verification Is Not Knowledge: Science needs more than answers
Jessica Hullman argues that verifiable AI outputs alone do not constitute scientific knowledge without human understanding and accountability.
VeriCUA verifies computer-using agents by gathering GUI and code evidence
The framework builds a task rubric, then runs GUI or code checks and rejects unsupported completion claims.
Independent review lifts coding-agent accuracy at modest extra cost
A critic review gains 5.9–9.5 points on Terminal-Bench and 3.9–6.6 on SWE-bench Verified at about baseline cost.
AgentProofBench tests reusable web-agent verifiers across 97 tasks
DeepSeek Flash and Pro keep most source programs under replay tests, while panel errors track failed deliveries.
Falcon uses execution traces to verify secure agentic code
The framework ties security rewards to real program behavior during test-time repair and reinforcement learning.
AI Can Write the Code. Who Verifies It?
A Faros study and tooling survey say AI coding raises bugs and incidents unless automated verification keeps pace.
20
Verification with Transfer: Exact Information Frontiers and Their Price in Calls
Theory paper prices source-task calls that cut verification cost, with Lean-checked bounds and a transformer test.
VeriFine: Scaling Verification for Self-Improvement in Embodied Reasoning
VeriFine co-evolves an embodied policy, curriculum, and judge so verification keeps pace with new failure patterns.
When Tools Lie: Reliability of Mathematical Agents Under Corrupted Tool Feedback
Corrupted tool feedback drops math-agent accuracy from 100% to 72.4%; mandatory reflection restores 100%.
VeriFine: Scaling Verification for Self-Improvement in Embodied Reasoning
VeriFine co-evolves policies, curricula, and judges so embodied agents keep self-improving as failure patterns change.
SoK: Semantic Decision Engines in Network Control Loops
A survey of 139 semantic decision-engine families finds almost none prove they meet network control-loop deadlines.
SoK: Decentralized Agent Economic Infrastructure
A systematization finds decentralized agent workflows can pass each check yet fail end-to-end verification-to-settlement guarantees.
Evidence Coverage for Intent-Bound Execution: Scope, Obligations, and Cutoff Reasoning
Authors define a cutoff-based model that judges whether execution-evidence obligations are complete, incomplete, or unknown.
Verify Claims, Not Scores: Evidence-Based Verification of Modular Agents
A new audit scores modular-agent conclusions from recorded evidence and verdicts instead of one aggregate task score.
Security Properties of Neural Networks as Decision Problems
Researchers classify neural-network security checks as co-NP-complete or harder, including backdoors and parameter faults.
Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents
Mid-Harness samples and verifies terminal-agent actions before execution, lifting Pass@1 on TerminalBench-Lite.
Classical Verification of Quantum Computation with Quasilinear Resources, from Compiled Nonlocal Games
Researchers give a classical verifier a quasilinear-cost argument system for BQP under LWE.
Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents
Multiverse Computing's Hugging Face blog proposes source-aware verification so MCP agents validate information provenance, not just factual correctness.
OPFL: Optimistic Verification of Federated Learning via Empirical Boundary
OPFL verifies federated learning inside MPC with an empirical gradient boundary, stopping model poisoning at much lower cost.
EdgeCraft: Automated Model Crafting for Edge IoT
EdgeCraft uses an LLM to craft deployable edge ML models that meet quality, latency, and energy goals.
When Valid Tool Calls Change Meaning: Formation-Consistent Dispatch for LLM Agents
Formation-consistent dispatch stops schema-valid LLM tool calls from gaining a different security effect after rollout or reconnect.
A2A-ForensicTrace: Offline Verification of Tamper-Evident A2A Runtime Evidence
A2A-ForensicTrace offline-verifies tamper-evident Agent2Agent evidence, and all 240 test receipts checked out.
A2A-CaseVerify: Merkle-Linked Case-Evidence Verification for Cross-Organization A2A Workflows
A2A-CaseVerify uses Merkle roots to check that agent workflow claims have complete evidence paths.