Ai2 open-sources ToyBlocks robot simulation framework

ToyBlocks is an open simulation framework built around a robot arm and blocks on a tabletop that automatically generates tasks and demonstrations, using reinforcement learning and motion planning to produce skills such as pushing, flipping, and stacking. It supports studies of low-level action model generalization, high-level planning with vision-language models, agentic systems using tool calls, action-conditioned world models, and zero-shot sim-to-real transfer, with an initial release of 85,336 long-horizon demonstrations covering nearly 1,600 hours and 144 million frames. The authors present ToyBlocks as the first of their FoundationWorlds and plan a second release with a larger dataset, more skills, and integrated RL training.

Ai2 introduces FoundationWorlds, a framework pairing controlled simulated environments with tools for automatically generating tasks, demonstrations, and behavioral metrics, starting with ToyBlocks, an open simulation of a robot arm manipulating blocks on a tabletop. ToyBlocks generates demonstrations via reinforcement learning, motion planning, and a task-and-motion planner at up to 24 times real-time on a single GPU, and supports studies of data scaling, spatial generalization, long-horizon planning, world models, and sim-to-real transfer. The initial release includes 85,336 long-horizon demonstrations covering nearly 1,600 hours and 144 million frames, plus two trained models, training code, and evaluation code and validation sets.