Skip to content
NewA calmer, faster ZeroHour

VAST compiles vulnerability scenarios into ranges for LLM red teams

Original title: VAST: Compiling Grounded Vulnerability Scenarios into Verifiable, Multi-Step Environments for Evaluating LLM Red-Team Agents

arXiv cs.CRby Zhen Wan ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

Budget-matched tests show 12 of 90 solves without the protocol and 76 of 90 when it is supplied in full.

VAST is a declarative range compiler that separates vulnerability scenarios from a workflow-hardening layer for LLM red-team evaluation. The retained benchmark holds 109 seed edges, 81 topic configurations and 1,387 instances. On TopicServer, three models solved 12 of 90 instances with contract-only prompts and 76 of 90 with the full workflow guide. A ranges_v2 ledger solved 88 of 126, and LoRA fine-tuning of Qwen2.5-Coder-7B-Instruct stayed low on 33 held-out instances.

Key facts
  • Forty-five seed edges carry strict upstream CVE annotations.
  • full_guide completion was 28/30, 23/30 and 25/30 across the three models.
  • Disclosing worker modules lifted Redis chaining solves from 0/18 to 18/18.
  • Header-Recovery SFT scored 4.67 of 33; Base scored 0/33.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

Evaluating Large Language Model (LLM) security agents requires reproducible environments whose workflow difficulty can be varied and whose failures can be diagnosed. We present VAST (Verifiable Adversarial Scenario Training & Evaluation), a declarative range compiler separating vulnerability scenarios from a compiler-introduced workflow-hardening layer.

arXiv cs.CR

By Zhen Wan · arxiv.org

Read the full article at arXiv cs.CR

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.