Efficient Reasoning Training Does Not Always Harm CoT Faithfulness and Monitorability
Efficient reasoning fine-tuning usually reduces chain-of-thought faithfulness, while monitorability stays more robust.
Researchers fine-tuned multiple models with three length-pressure methods: a fixed generation budget, a per-example length target, and a group-relative length reward. Faithfulness, how well the chain of thought reflects decisions on related inputs, fell in most settings mainly because the models became less consistent. Monitorability was more robust: models still acknowledged when input interventions changed the output, even with much shorter reasoning.
- Fixed budgets, per-example targets, and group-relative rewards apply length pressure differently.
- Faithfulness fell in most settings, mainly because models became less consistent.
- Monitorability stayed robust even when chains of thought were much shorter.
- Models still acknowledged when input interventions changed their answers.
From the original article
1 min read at huggingface.co
Chain-of-thought (CoT) reasoning allows humans to inspect how large language models reach their answers, and oversee model behaviour. This reasoning comes at an increased inference cost, motivating efficient methods that train models to solve tasks using fewer tokens.
By Samuel Lewis-Lim, Xingwei Tan, Mario Sanger · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.