More Performance From Every Byte: How Full-Stack Codesign Solves AI’s Memory Challenge

NVIDIA describes how its full-stack codesign approach addresses growing GPU memory pressure from larger models, longer contexts and agentic applications by designing GPUs, CPUs, networking, storage and software together across a memory hierarchy. Techniques include NVFP4 for model weights and KV cache, distributing weights across up to 72 GPUs in NVL72, offloading inactive context to CMX context memory storage via Dynamo, and keeping RAG indexes outside HBM with Nemotron Retriever and cuVS. NVIDIA reports that from Blackwell Ultra to Vera Rubin, HBM capacity per GPU rose roughly 3% while agentic throughput per dollar rose up to 30x on SemiAnalysis AgentX.