When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents
ToxicBench finds poisoned tool outputs cut data-agent success 26 to 39 points, with wrong answers adopted after checks.
The paper studies tool-augmented data agents that receive plausible but incorrect tool evidence and must decide whether to trust or verify it. ToxicBench pairs clean and poisoned observations over fixed source data across numerical, label, schema, and retrieval errors. In a 118-task GPT evaluation across three adapters, poisoning lowers task success by 26 to 39 percentage points. Retries help against one-shot poisoning, but repeated poisoning leads agents to adopt wrong answers even after checking; human review of 200 trajectories agrees with the scorer on task success 96 percent of the time.
- ToxicBench covers numerical, label, schema, and retrieval errors
- Poisoning lowers GPT task success by 26 to 39 points
- Repeated poisoning leads to wrong answers after checking
- Human labels match the scorer on 96 percent of 200 trajectories
From the original article
1 min read at arxiv.orgTool-augmented data agents rely on tool outputs for analytical decisions. Yet successful execution can return plausible but incorrect evidence, requiring agents to decide whether to trust or verify it. Understanding this failure requires examining both the evidence obtained through checking and the answer ultimately adopted.
By Zifu Tao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.