Shibboleth traps fingerprint LLMs and harnesses in agentic attacks
Original title: Say Shibboleth: Fingerprinting the LLMs and Harnesses Behind Agentic Attacks
Combined inference reached 90% top-1 accuracy across 15 models and 92% across four harnesses.
Researchers propose shibboleth traps, modular honeypot services that elicit distinctive behavior from agentic attackers without a successful prompt injection. Separate probes infer how a harness masks secrets, enforces command time limits, and schedules tool calls from later requests. Combined inference distinguished 15 candidate models at 90% average top-1 accuracy, and fixed rules identified the harness among four with 92% accuracy.
- Traps require informative interactions before an agent can obtain sought resources.
- Behavioral classifiers were tested on held-out harnesses or task families.
- Agents instructed to conceal identity still left useful behavioral evidence.
- Their declarations became uninformative under those concealment instructions.
From the original article
1 min read at arxiv.orgWe study the fingerprinting of agentic cyberattacks. Our objective is to reliably infer information about the attacker's configuration, such as LLM version and harness, from telemetry. To achieve this, we introduce a technique we call shibboleth traps: modular honeypot services designed to elicit discriminative behaviors from agentic attackers.
By Dario Pasquini · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.