When Models Edit Too Much: On the Fidelity of Minimal Code Edits
A 400-task BigCodeBench evaluation shows frontier LLMs widely over-edit code; a preservation instruction cuts excess edits and raises Pass@1 by 2.3 points.
Researchers built an evaluation framework from 400 BigCodeBench problems with injected AST-level corruptions, each with a known minimal patch, to measure over-editing in LLM code repair. Even strong models like GPT-5.5 produce unnecessarily large edits despite high Pass@1. Adding a preservation instruction reduced average excess Levenshtein distance from 0.195 to 0.131, cut added cognitive complexity by 26.6%, and raised Pass@1 by 2.3 points. Reinforcement learning post-training gave the best out-of-domain edit-fidelity trade-off, while supervised fine-tuning overfit to seen corruption patterns.
- Evaluation injects AST corruptions into 400 BigCodeBench tasks with known minimal patches
- Over-editing is widespread even in frontier models like GPT-5.5
- Preservation instruction cuts excess edit distance from 0.195 to 0.131 and raises Pass@1 2.3 points
- RL beats supervised fine-tuning for out-of-domain edit fidelity; SFT overfits seen corruptions
- Products
- BigCodeBench
- AI models
- GPT-5.5
From the original article
1 min read at huggingface.co
Large language models (LLMs) are increasingly used to edit existing code, but correctness alone is not enough: useful repairs should also be minimal, reviewable, and faithful to the original implementation. We study over-editing, the tendency of a model to rewrite code beyond what is required to fix a bug.
By Tongyao Zhu, Wei Hern Lim, Min-Yen Kan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.