Stanford Research Open-Sources Memorilla, a Latent Memory Module for Frozen LLMs

Stanford researchers have released Memorilla, an open-source 194M-parameter memory module that condenses a whole document collection into 16 memory tokens for a frozen LLM, with code, training recipes and checkpoints public under Apache 2.0. On FactKG, a frozen Qwen3-8B scored 84.33% with Memorilla versus 80.43% for full context, using 108x fewer document tokens, and it reported the highest average (49.68) across 8 benchmarks while losing on PubMedQA (59.00% vs 69.00% for LLMLingua-2). The quickstart requires one 80 GB CUDA GPU, and the full training recipe takes about 19 hours on 4 H100 GPUs.