Meta: A Preliminary Analysis of the Impact of AI Asst'd/Gen'd Code [..] at Scale
Meta: A Preliminary Analysis of the Impact of AI Asst'd/Gen'd Code [..] at Scale
Using data from over 10K developers and 400K diffs at Meta with character-level provenance tracing of AI-suggested code, the authors model which context factors predict AI code landing, how landed AI code relates to review time, and its association with production outages (SEVs) while controlling for contextual confounds. They find AI code lands more often in less central code, larger changes, test files, and for authors with higher tenure, and that while review time decreases and SEV rates are lower for diffs with AI code, both associations are confounded by AI code appearing in less central contexts. The authors conclude that naive comparisons of AI vs. non-AI work may reach misleading conclusions and offer guidance for controlling these confounds.
Source: Hacker News AI · drops.dagstuhl.de