VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes
VidaForge releases open infrastructure and VIDAFORGE-3M (3.14M clips, 6,475 hours) linking video pretraining data recipes to downstream model performance.
VidaForge is an open research infrastructure that represents a video pretraining data recipe as an executable five-stage workflow from raw videos to training datasets. The team compares data recipes with different coverage and quality during early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1, finding that broader-coverage recipes achieve the highest downstream benchmark scores while loss-based evaluation favors different recipes. They also release VIDAFORGE-3M, containing 3.14 million scene-level clips totaling 6,475 hours with fine-grained annotations and curation signals for video data-recipe research.
- Represents video data recipes as executable five-stage workflows with reproducible sample provenance
- Broader-coverage recipes achieve the highest downstream scores in Wan 2.1 and V-JEPA 2.1 pretraining
- Releases VIDAFORGE-3M: 3.14M scene-level clips, 6,475 hours, with annotations and curation signals
Full article170 words · extracted from huggingface.co · click to collapse
Video foundation models increasingly rely on large-scale pretraining data, yet the end-to-end data pipelines behind them remain largely closed and difficult to inspect or reuse. Researchers seeking to understand how video data recipes affect model pretraining often need to build substantial infrastructure before testing even a focused hypothesis. We present VIDAFORGE, an open research infrastructure that represents a video data recipe as an executable five-stage workflow from raw videos to training datasets. A decision in this workflow can be varied to construct alternative datasets while preserving how every sample was produced. To demon strate this research workflow, we compare data recipes with different coverage and quality in early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1. Across both learning objectives, the broader-coverage recipe achieves the highest downstream benchmark scores, while loss-based evaluation favors different recipes. This study demonstrates how VidaForge connects data-recipe choices to downstream model performance. We further release VIDAFORGE-3M, containing 3.14 million scene level clips totaling 6,475 hours, with fine-grained annotations and curation signals for video data-recipe research.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.06652