Datadog tests security rules on coding agents' secure code
Original title: Putting models to the secure coding test: Security rules
Without rules, Sonnet 5, Composer 2.5, and GPT 5.5 all left IDOR flaws in a document portal.
Datadog Security Labs asked whether Project CodeGuard rules change how coding agents write a small document portal. Without rules, Sonnet 5, Composer 2.5, and GPT 5.5 each let any authenticated session access or annotate other users' files. The provided text ends before the with-rules results.
Unconfirmed: With-rules results are missing from the excerpt, and each model was run only once.
- Project CodeGuard v1.4.0 was installed for Claude Code; Cursor and Codex used zip installs.
- Each of the three models was tested once with the same prompt.
- Composer 2.5 used a hardcoded JWT_SECRET fallback and stored reset tokens in plaintext.
- GPT 5.5 had a CSRF issue in uploads and trusted client-provided MIME types.
- Sonnet 5 added magic-byte checks and double-submit CSRF but still missed document access scoping.
- Organizations
- Datadog
- AI models
- Sonnet 5Composer 2.5GPT 5.5
From the original article
12 min read at securitylabs.datadoghq.com
We tested Sonnet 5, Composer 2. 5, and GPT 5. 5 using set security rules to see whether using them produces measurably more secure code.
By @datadoghq · securitylabs.datadoghq.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.