ShadowBench-MCP measures shadow invocations in MCP-style tool routing
Original title: ShadowBench-MCP: Measuring Shadow Invocations in MCP-Style Tool Routing
On a Groq-hosted qwen3-32b router, efficiency-first routing had a 29.2% shadow-invocation rate versus 4.2% when policy-aware.
ShadowBench-MCP scores task-preserving request variants that make an LLM tool router choose a riskier path and skip validation, provenance, staging, redaction, or audit controls. A policy-ablation study used a Groq-hosted qwen/qwen3-32b router across six security-sensitive workflows. Policy-aware routing sharply reduced both shadow invocations and safety-tool skips. The authors present this as early single-router evidence, not a universal vulnerability.
Unconfirmed: The authors say the findings are early evidence from one router, not a universal vulnerability.
- 120 routing decisions yielded 96 original-versus-variant comparisons.
- Efficiency-first routing skipped safety tools in 26.7% of cases, versus 1.7% when policy-aware.
- The relative reduction in shadow invocations was 85.7%.
- Pre-execution signals distinguished shadow routes with ROC AUC up to 0.94.
- Products
- ShadowBench-MCP
- Organizations
- Groq
- AI models
- qwen/qwen3-32b
From the original article
1 min read at arxiv.orgLLM agents increasingly act as routers for tools, APIs, and software-supply-chain workflows, making an execution path - rather than generated text alone - a security-relevant model output.
By Eyhab Al-Masri · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.