Jev Is Not a Language Model, but It Breaks Like One
Check Point shows TypeSafe AI's Jev decision model can be prompt-injected into flipping verdicts for about 50 cents.
Check Point tested Jev, a TypeSafe AI model that returns typed decisions instead of text, in a due-diligence assistant reviewing a fictional high-risk company, PonziCorp. Attackers controlling one section of the uploaded report flipped the verdict to low risk and an invest recommendation in every configuration. The strongest attacker succeeded in 25 of 27 runs, typically on the fourth turn, at about 50 cents per break. Marking the document untrusted or adding anti-injection instructions made little difference, while reasoning effort improved resilience on comparison models.
- Every tested configuration let attackers flip Jev's investment verdict.
- The strongest attacker succeeded in 25 of 27 runs, about $0.50 each.
- Untrusted labels and anti-injection instructions barely changed results.
- Reasoning effort was the strongest defense on comparable models.
- Jev returns typed decisions intended for software rather than human review.
- Vendors
- TypeSafe AICheck Point
- Organizations
- Check PointTypeSafe AI
- AI models
- Jev
From the original article
7 min read at blog.checkpoint.com
A new kind of AI model returns decisions instead of text. We spent a day trying to change its mind. It cost about 50 cents.
By patresh · blog.checkpoint.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.