When AI Fails: Building Fault-Tolerant AI Systems

Aman Sardana argues that AI components should be treated as dependencies that can fail, and that classic reliability patterns—timeouts, circuit breakers, bulkheads, retries, fallbacks, and graceful degradation—must be extended to cover AI's quiet failures, where a model returns a fast, well-formed, confident but wrong answer. He proposes a semantic circuit breaker that trips on validation failures and low confidence, layered response validation, a deliberately designed fallback chain, and a pre-agreed degradation table specifying fail-open or fail-closed behavior for each use case.