Audit: 296 of 363 LLMs leaked system-prompt secrets
Original title: Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study
An OpenCode agent using DeepSeek V4.1 Flash extracted 16-character flags, often after a French language pivot.
Researchers audited instruction-hierarchy enforcement on 363 text-in/text-out models from the OpenRouter catalog. Each model guarded a 16-character flag in its system prompt while an OpenCode agent, powered by DeepSeek V4.1 Flash and steered by a human, tried to elicit it. The flag was taken from 296 catalog models plus four later releases, 300 captures in total; 170 used a non-English pivot, most often French. The authors say a capture shows one ranking of a user request over the system instruction, not resistance to untested attacks or harmful-content jailbreaks.
Unconfirmed: The study does not claim models would consistently leak semantically sensitive data or fail harmful-content jailbreaks.
- 296 of 363 catalog models leaked the flag, plus four models released after the catalog freeze.
- 170 of the captures used a non-English language pivot, most often French.
- Authors advise against storing secrets in system prompts and against relying on an LLM judge alone.
- They recommend output validation and deterministic checks outside the model.
- Products
- OpenCode
- Organizations
- OpenRouter
- AI models
- DeepSeek V4.1 Flash
From the original article
1 min read at arxiv.orgWe audit instruction-hierarchy enforcement across 363 text-in/text-out models from the OpenRouter catalog. The research question we answer with this experiment is which models disclose a protected secret embedded in the system prompt, and through which techniques. Each model guards a secret 16-character flag set in its system prompt, while an artificial intelligence (AI) agent (OpenCode, powered by DeepSeek V4.
By Ana Clara Zoppi Serpa · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.