EmoHarness raises small multimodal emotion scores with audio evidence
Original title: EmoHarness: Boosting Multimodal Emotion Understanding with Meta-Harness Evolving
On 5,507 MME-Emotion items, Recognition rose 7.6-13.4 points, mostly from transcripts and prosody rather than evolved edits.
Researchers present EmoHarness, a harness around a frozen small multimodal student that cites visual, transcript and prosody evidence when judging emotion. A stronger Builder LLM edits the harness offline from errors and traces, with checks and validation gates, and never sees evaluation items. Gains come chiefly from audio-derived evidence; evolved edits add at most 0.95 points and do not survive Holm correction. Small IntentQA transfers also fail correction.
- Transcripts and prosody descriptions add +5.38 to +10.71 points.
- The scaffold changes the audio-augmented call by -0.79 to +2.54.
- Evolved edits add at most +0.95 and fail Holm correction.
- IntentQA gains for three of four students also fail correction.
From the original article
1 min read at huggingface.co
Small multimodal models can misjudge emotion when words, voice, and visual cues conflict. We present EmoHarness, which runs such a model, the frozen student, inside a harness: the prompts, settings, and verification code around its calls.
By Fenghai Li, Bingxuan Li, Jiawen Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.