τ-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions
τ-Emotion-Bench shows emotional users expose reliability gaps in tool-calling LLM agents.
Researchers introduce τ-Emotion-Bench, a ten-domain benchmark that evaluates tool-calling LLM agents while varying user emotion without changing the task objective. Tracer, an agentic data pipeline, builds grounded policy-valid tasks and collects trajectories through an interactive agent-user-environment harness. Allowing dissatisfied users to terminate inefficient interactions filters trajectories with excessive user dependence. Experiments with two user simulators find frontier agents unreliable, while Tracer supervision improves Qwen3 across scales, generalizes to unseen domains, and transfers to τ-Bench and τ-Trait.
- The benchmark covers ten domains with emotion-conditioned but goal-preserving users.
- Tracer generates policy-valid tasks and interaction trajectories at scale.
- Users can end inefficient sessions, penalizing excessive user dependence.
- Tracer supervision improves Qwen3 and transfers to τ-Bench and τ-Trait.
- AI models
- Qwen3
From the original article
1 min read at huggingface.co
In long-horizon tool-calling tasks, LLM agents can fail not only because of tool-use errors, but also because of ineffective interaction with users. Yet existing benchmarks largely assume cooperative and behaviorally stable users, leaving the effect of emotional variation on agent reliability underexplored.
By Rishitosh Kumar Singh, Amir Saeidi, Shamanthak Hegde · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.