Meta puts LLM-generated testing system JiTTests into production
Understand this story
Meta has moved its LLM-based JiTTests from a research prototype into production infrastructure. The system generates tests that should fail against pending code changes (diffs), in order to prove that the change actually alters behavior. The Meta team says that in the first half of 2026 it caught 74 privacy SEVs at the diff stage, while the privacy SEV rate fell 20% over the same period; engineers self-report a true positive rate of 60% under full deployment. JiTTests runs on Meta's internal infrastructure, using idle overnight compute, and is not publicly available; Meta has not released code, models, or APIs.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- MarkTechPostMeet Meta’s JiTTests: LLM-Generated Tests Designed to Fail Before Risky Code Lands
Meta has moved JiTTests, an LLM-based system that generates executable tests designed to fail on risky code changes, from research prototype into production infrastructure, as described in the FSE 2026 paper 'Just-in-Time Catching Test Generation at Meta.' The team reports that JiTTests caught 74 privacy SEVs at the diff stage in the first half of 2026, with privacy SEV occurrence down 20%, and engineers report a 60% true-positive rate at full scale, up from 19.5% in the initial deployment. The system is internal only, with no code, models or API released.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 16 19:00
- Change in the past 24 hours
- -50%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.