Octopus measures AI's impact on delivery with engineering metrics
Understand this story
Octopus has relaunched its engineering "vital signs" metrics system, using quantifiable data such as delivery and stability to answer whether AI has truly improved software build efficiency. The company says that the AI software factory's claim of "maximizing valuable software" is only a vision rather than a measurable goal; with the number of engineers essentially unchanged, monthly merged PRs more than doubling and the time spent on major and critical incidents dropping by about 70% are evidence closer to the facts. These data reflect changes Octopus itself observed during the same period, not independently verified causal effects of AI.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIEngineering Vitals in the Age of AI
An Octopus CTO argues that AI-era engineering measurement should focus on outputs that can be steered week to week—deployment lead time, PR churn and review latency, time to recovery, build health, batch size, Sev 0-2 defect counts, and R&D NPS—rather than lagging value metrics or activity counts. The post explains why Octopus dropped change failure rate in favor of counting high-severity defects reaching customers, and treats pull request volume as a trend without a target, citing DORA, Pragmatic Engineer, and Atlassian DX data on rising throughput and PR size.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 18 12:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.