Accurate but Not Humble: Evaluating Epistemic Humility in LLM Agents under Knowledge Conflict

Source facts

The preprint evaluates epistemic humility through three trajectory-level behaviors: Identify, Solve, and Escalate. It tests controlled and naturally occurring conflicts between parametric knowledge and retrieved evidence, along with matched no-conflict controls. Across four agents, the authors report that task accuracy does not necessarily imply humility; agents may detect conflicts early but fail to preserve or communicate unresolved uncertainty in final answers. They report that interventions can improve humility at a cost to task accuracy.

Analysis

The work treats uncertainty handling as a trajectory capability rather than a single answer attribute. This makes the interaction among backbone model, harness, and environment visible, but the operational definition of humility still depends on the selected conflict tasks and evaluation judgments.

Uncertainty/Doubt

• This is an arXiv preprint. The conflict construction, agent configurations, and reported accuracy trade-offs were not independently reproduced here.