MiST: Mid-Training LLMs for Cybersecurity
MiST introduces 8B and 32B cybersecurity-specialized LLMs that outperform Qwen baselines by up to 13.1 points on public security benchmarks.
MiST (Mid-trained Security Transformer) applies mid-training as an intermediate adaptation stage, converting an expert-vetted seed corpus into high-quality synthetic domain data rather than continual pretraining on raw text. The 8B and 32B checkpoints improve mean cybersecurity accuracy by +13.1 and +8.6 absolute points over Qwen baselines (+27.0% and +15.8% relative). Ablations show gains arise in mid-training and supervised fine-tuning, and MiST provides stronger initialization for downstream fine-tuning and reinforcement learning.
55