Simple Puzzle Reveals AI's Jagged Intelligence

An article describes training a GPT from scratch on a million solutions to the 15 puzzle, after which it solves 99.9 percent of randomly scrambled boards but fails on the easiest puzzles, including one that is already solved, while assigning its wrong moves over 97 percent probability. The author attributes this to AI emulating its training data through shortcuts rather than understanding, producing a "jagged" boundary where performance collapses outside the training distribution. The piece draws on Ilya Sutskever and Melanie Mitchell to argue that skill is not understanding and that AI can be confidently wrong.