EquivSVA: A Formally Verified Dataset of Behavioral Assertions Across Equivalent RTL Implementations
EquivSVA provides 120 verified RTL behavior families to test whether LLM assertions capture observable behavior.
EquivSVA is a formally verified dataset for studying whether LLM-generated SystemVerilog assertions capture externally observable behavior or incidental details of one RTL implementation. It contains 120 behavior families across 12 categories, 480 reference RTL implementations, 914 gold properties, and 360 controlled mutants, and every final family passes a 17-job validation suite. On the held-out test split, Qwen2.5-Coder-7B-Instruct produced 293 interface-only properties, of which 93 were formally sound. Soundness varied across equivalent implementations for 14 of 24 test families. The dataset, generators, and validation scripts are public.
- 120 behavior families, 480 RTL designs, 914 gold properties, 360 mutants.
- Every final family passes a fixed 17-job formal validation suite.
- Qwen2.5-Coder-7B: 93 of 293 generated properties formally sound.
- Sound property counts vary across equivalent RTL in 14 of 24 families.
- AI models
- Qwen2.5-Coder-7B-Instruct
From the original article
1 min read at arxiv.orgLarge language models are increasingly used to generate SystemVerilog Assertions from natural-language specifica- tions and register-transfer-level designs. Existing datasets and benchmarks support important goals such as large- scale training, formal evaluation, specification-to-assertion generation, and mutation-based testing.
By FNU Aditi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.