Closed-World Resolution Against Tool Hallucination in LLM Agents

Benchmark across ten LLMs documents 322 tool hallucinations and 154 more on MCP, showing model scale does not help and gates cannot reject fabricated calls.

The paper presents a five-class taxonomy (H1-H5) of tool hallucination in LLM agents, where models call nonexistent tools or pass arguments no schema declares, a blind spot no gating defense can reject since no gate made the decision. Across ten hosted models on two invocation surfaces, researchers measured 322 genuine hallucinations, concentrated on the unconstrained raw-JSON surface (34 versus 3), with model scale offering no benefit as a 675B model matched a 7-8B one. On the Model Context Protocol, merging servers into one namespace produced 154 hallucinations, including from frontier models that were clean on the single-registry surface. The versioned Hallucinated-Tools Benchmark (HTB) is released for comparable resolver evaluation.