Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict
High-accuracy LLM agents often detect knowledge conflicts but fail to communicate unresolved uncertainty.
The paper evaluates epistemic humility in LLM agents: whether they recognize, act on, and communicate uncertainty when retrieved evidence conflicts with parametric knowledge or with another source. Humility is operationalized as Identify, Solve, and Escalate behaviors, tested under controlled conflicts and naturally occurring conflicts during multi-step execution, each with matched no-conflict controls. Across four agents, higher task accuracy did not reliably mean greater humility; some configurations detected conflicts early but still failed to report unresolved uncertainty in incorrect final answers. Model-level interventions improved humility, often at the cost of task accuracy.
- Epistemic humility is scored as Identify, Solve, and Escalate along agent trajectories.
- Tests cover controlled conflicts and natural conflicts in multi-step execution.
- Four agents show accuracy does not imply communicating unresolved uncertainty.
- Agents often detect conflicts early but fail to maintain or resolve them later.
- Model interventions can raise humility while reducing task accuracy.
From the original article
1 min read at huggingface.co
When retrieved evidence contradicts an agent's prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts.
By Kaiser Sun, Bernal Jimenez Gutierrez, Hongjun Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.