Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
Manacá-1B-Instruct-2 gained safety from iterated safety finetuning after three failed DPO runs, beating four Portuguese baselines.
The paper presents Manacá-1B-Instruct-2, the second safety-aligned release of Manacá-1B, a 1.72B-parameter model trained from scratch for Brazilian Portuguese using 291,223 supervised-finetuning examples. Three DPO attempts were negative, including one where the training reward margin reached 1.0 while free generation did not change. Safety gains came from iterated safety supervised finetuning; mixing 10% to 60% general data produced a non-monotonic over-refusal curve with a knee at 40-50%. Its safety interval lower bound of 94.1% exceeded four Portuguese baselines, and an LLM judge refused to score two of six MT-Bench-PT safety items.
- Manacá-1B has 1.72B parameters and 291,223 supervised-finetuning examples.
- Three DPO attempts failed, including a 1.0 reward margin with unchanged generation.
- Safety-data mix showed non-monotonic over-refusal, with a knee at 40-50%.
- Its 94.1% safety lower bound beat four Portuguese baselines under corrected tests.
- An LLM judge refused to score two of six MT-Bench-PT safety items.
- Organizations
- LLM-jp
- Countries
- Brazil
- AI models
- Manacá-1B-Instruct-2Manacá-1B
From the original article
1 min read at arxiv.orgBrazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or validate comparisons with statistical significance. We present Manacá-1B-Instruct-2, the second-generation post-training alignment of Manacá-1B (1. 72B parameters, trained from scratch for PT-BR), with an SFT corpus of 291,223 examples.
By Bruno Leonardo Santos Menezes · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.