NHNM mines natural hard negatives for instruction retrieval
Fine-tuning 568M to 4B models on nearly 7K NHNM-MARCO instances matches or beats much larger retrievers on InstructIR and FollowIR.
Stories per day · 30 days
2 RSS
Relatedbenchmarks2agents1alignment1contrastive-learning1evaluation1gpt-4o-mini1hard-negatives1honesty1information-retrieval1language-models1memory-evaluation1model-safety1msmarco1video-world-models1weight-editing1
12 stories, newest first · one card per story
Stories tagged #instruction-following
Instruction capacity metric measures how models hold rules under load
On GPT-4o mini, capacity was about 37-38 rules at 0.95 accuracy and 18-19 at full replacement density.
MIST finds video world models fail to execute changes, not only memory
M3-Agent more than triples the MIST reveal score by revising instructions so a frozen model executes the change.
Training-free weight edits reduce instructed lying and sexism
Honest responses rise 6.2 and 4.6 points; sexist responses fall from 57.9% to 24.3%, or 11.9% with a fairness prompt.
Thinking Inertia: LLMs Keep Thinking When Told Not To
Study finds LLMs keep producing explicit reasoning even when instructed not to think.
Your Prompt Should Do More: Effects of Retrieval Instructions in Embedding Models
Embedding models often ignore simple retrieval instructions when query-side distractors appear, but targeted fine-tuning largely fixes this.
Enhancing Diffusion Language Models with Autoregressive Post-Training Weights
A2D recycles autoregressive post-training weights to improve diffusion language models without extra training.
You Changed Your Mind, The Model Didn't: Demystifying Intent in Multi-Turn Dialogue
Intent-Eval shows rejected or replaced user changes can still derail multi-turn LLM task execution.
Gacha Decoding: Eliciting Diverse Generations Through Instruction Following
Gacha Decoding uses instruction following and an external RNG to elicit diverse, high-quality language-model generations.
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
Elastic Weight Consolidation limits general forgetting in translation fine-tuning but not translation-specific instruction following.
RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents
RoboFollow shows embodied agents' high success rates mask weak genuine instruction following.
Claude is a Contrarian
Opinion piece argues Claude habitually contradicts explicit user instructions, injecting contrarian content despite CLAUDE.md rules and user objections.