MLLM framework detects hidden emotions with temporal context
Original title: Beyond Multimodal Emotion Conflict: Temporal Context-Aware Hidden Emotion Detection with Multimodal Large Language Models
The work builds a benchmark with expressed, internal emotion and concealment annotations to distinguish ordinary inconsistency from concealment using MLLM.
Multimodal emotion recognition combines textual, acoustic and visual signals but conflicts may indicate mixed emotions or context rather than concealment. This paper proposes distinguishing these cases and inferring underlying emotions when evidence supports it. Researchers plan a multimodal benchmark with explicit annotations and an MLLM framework integrating facial Action Units, cross-modal features and temporal context.
- Constructs benchmark with expressed, internal emotion and concealment annotations
- Develops MLLM-based framework using facial Action Units
- Integrates cross-modal features and temporal context
- Distinguishes ordinary inconsistency from concealment
From the original article
1 min read at arxiv.orgMultimodal emotion recognition commonly combines textual, acoustic, and visual signals to infer affective states. Recent work has begun to study conflicts among modalities, but a conflict does not necessarily indicate that a person is concealing an emotion. It may also result from mixed emotions, individual expressive styles, or contextual factors.
By Yonghao Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.