StateBench: Testing how language models track state changes over long documents

Researchers introduce StateBench, a benchmark with four difficulty levels of long-context state tracking, from immutable assignment to general finite problems involving assignment by reference. In tests, open pretrained models and frontier models (Gemini 3.8 Flash, Claude Opus 5.5, GPT-6 Sol and Luna) degraded on harder levels as inputs grew, while 275M-parameter Gated DeltaNet and hybrid models trained directly on the tasks generalized far beyond training length, though transformers never learned the hardest level. The work is released with a tech report, data, and code, and notes open questions such as fixed variable counts and the exact hierarchy placement of constrained Gated DeltaNet variants.