Skip to content
NewA calmer, faster ZeroHour

Meta runs JiTTests to fail risky diffs before code lands

Original title: Meet Meta’s JiTTests: LLM-Generated Tests Designed to Fail Before Risky Code Lands

MarkTechPostby Michal Sutter ()8 min read1
AI summary · x-ai/grok-4.7Summaries by Vani AI

Meta says the internal system caught 74 privacy SEVs in the first half of 2026, with a self-reported 60% true-positive rate.

Meta has moved Catching Just-in-Time Tests from a research prototype into production. JiTTests uses LLMs to write executable tests meant to fail when a diff changes behavior, then ranks failures with a judge ensemble and rule checks before contacting engineers. A paper analyzes 22,126 generated tests on Meta backend code. The system is internal only, and the production figures are Meta’s own.

Unconfirmed: Catch rates and the 60% true-positive figure are self-reported, and a weak catch is not a confirmed bug.

Key facts
  • Intent-aware tests produced weak catches in 6.4% of tests and on 7.9% of diffs.
  • An early deployment confirmed 8 of 41 reported catches, a 19.5% rate.
  • Judge models named in the paper are Llama3.3-70B, Gemini 3 Pro and Claude Sonnet 4.
  • 589 engineers engaged, 64% of those contacted, according to an October 2026 post.
  • Privacy SEV occurrence was reported down 20%.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

8 min read at marktechpost.com

Meta has moved Catching JiTTests into production. The LLM system writes tests designed to fail on risky diffs, and caught 74 privacy SEVs before landing in H1 2026, with a 60% engineer-reported true-positive rate. The post Meet Meta’s JiTTests: LLM-Generated Tests Designed to Fail Before Risky Code Lands appeared first on MarkTechPost.

MarkTechPost

By Michal Sutter · marktechpost.com

Read the full article at MarkTechPost

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.