Transcript Testing: Helping Humans Verify AI Code

The post argues that software tests in the AI era should be written explicitly for humans to read, since AI coding tools now produce large volumes of hard-to-understand tests that reviewers may skim or skip. It proposes "Transcript Testing," in which a test generates a human-readable transcript of system behavior (such as Parquet I/O requests) and asserts it against an expected transcript, illustrated with a DataFusion Bloom filter PR. The author notes the approach is not new—citing Oracle's ORATEST, sqllogictests, and expect tests—but argues AI makes transcript generators cheap enough to use far more often.