arXiv studies training data for open models on ARC-AGI-2
Original title: Rules, Traces and Routing Statistics: Training and Calibration Data for Open Models on ARC-AGI-2
Fine-tuning open models with specific rules and traces improves scores on ARC-AGI-2, with the best reaching 56.67.
The paper studies which training and calibration data help three open models solve ARC-AGI-2 tasks when run offline on four L4 GPUs for at most 12 hours. For the NVARC Qwen3-4B model, a corpus of 12,000 executable rules and 95,993 tasks was built; continued fine-tuning on 1,920 real task records raised the semi-private score from 28.96 to 33.06. For the Qwen3.8-27B model, reasoning traces raised the score from 26.81 to 44.03. For the Qwen3.8-Flash-Next 125B MoE model, pruning experts and using routing statistics reached 56.67.
- Corpus of 12,000 rules and 95,993 tasks for Qwen3-4B
- Fine-tuning on 1,920 real tasks raised score to 33.06
- Reasoning traces boosted Qwen3.8-27B to 44.03
- Pruning and routing statistics for 125B MoE reached 56.67
- No improvement from synthetic tasks
From the original article
1 min read at arxiv.orgWe study which training and calibration data help three open models solve ARC-AGI-2 tasks when the solver runs offline on four L4 GPUs for at most 12 hours. We compare data offline on public tasks and report 189 runs on the 120 hidden semi-private tasks.
By Nguyen Minh Thang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.