BypassGym tests whether AI agents can evade deployed EDR
Original title: BypassGym: A Benchmark for Evaluating AI Agents' Endpoint Defense Evasion Capabilities
DeepSeek-V4.1 Flash succeeded on 17 of 18 runs; Bitdefender blocked all but 5 of 15.
BypassGym is a benchmark that asks an AI agent to compile a loader for given command-and-control shellcode so the payload runs without an endpoint detection and response product blocking or terminating it. Evaluation uses Proxmox VE virtual machines with real Windows systems and EDR products and returns structured detection feedback. Five lightweight model backends were tested against six EDR products and three payloads. The authors report successful submissions even from simple models and say the results point defenders to areas to investigate.
Unconfirmed: Success counts are the authors' benchmark results; the claim of being the first such benchmark is theirs.
- A supplementary evasion-knowledge skill supports with-and-without comparisons.
- The environment returns structured detection feedback to the agent.
- Five model backends, six EDR products, and three payloads were tested.
- Bitdefender allowed 5 of 15 runs to pass, the paper reports.
- Vendors
- Bitdefender
- Products
- BypassGymProxmox VE
- AI models
- DeepSeek-V4.1 Flash
From the original article
1 min read at arxiv.orgArtificial intelligence agents increasingly automate cybersecurity tasks, including offensive operations. Frequent attacks across the web make endpoint detection and response (EDR) a critical defense, but whether AI agents can evade it remains unclear. Benchmarks evaluate the offensive capabilities of large language model (LLM) agents in areas such as penetration testing and vulnerability discovery.
By Weifeng Yuan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.