Datalab Introduces OmniExtractBench to Fix Bias and Opacity in Extraction Benchmarks
Datalab released OmniExtractBench, an open 620-document benchmark scoring PDF-to-JSON extraction with auditable verdicts.
Datalab released OmniExtractBench, an open benchmark that scores how accurately systems fill a JSON schema from a PDF. It combines 620 documents from four existing suites and grades every value with one deterministic scorer that emits six verdict types. Content-based row alignment and dropping null addresses are meant to stop positional table failures and schema-padding from distorting scores. On the full corpus, Datalab’s accurate mode led at 93.85 accuracy, with Datalab balanced at 93.48 and Reducto deep_extract v2 at 93.47.
48