Dynamic Conformance Testing of WebGPU Through Specification-Driven Mutation
Research paper introducing LANTERN, a dynamic conformance testing framework to uncover WebGPU browser bugs.
Stories per day · 30 days
2 RSS
4 stories, newest first · one card per story
Stories tagged #specification
Specification Before Generation: A Pre-Registered, Five-Model Paired Evaluation of a Specification Frame for LLM-Generated Code in Money, Time, Idempotency, and Access Tasks
A 267-word specification frame cut security defects in code from five frontier models, and never made any model worse.
Why I'm still bearish on LLMs after Navier-Stokes
Essay argues frontier LLMs remain far from autonomous knowledge-worker replacement because reward hacking and specification costs limit reliability to narrow, well-specified domains.
IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications
IdeaAMBIG benchmark of 660 specification-gap instances shows LLMs localize implementation-critical research gaps poorly, with best model at 9.6% defect recovery.