NVIDIA tackles AI memory pressure with full-stack co-design
Understand this story
NVIDIA says it alleviates AI memory pressure through full-stack co-design: from Blackwell Ultra to the Vera Rubin platform, HBM capacity per GPU increases by only about 3%, yet agent throughput per dollar on SemiAnalysis AgentX improves by up to 30x. The core of this comparison is that NVIDIA focuses on co-optimizing chips, systems, and software rather than simply expanding per-card memory capacity. The 30x improvement is a SemiAnalysis AgentX test result cited by NVIDIA, not a universally verified effect.
AI overview based on source reports · Updated 13 hr ago
Coverage timeline
Follow the coverage to explore different perspectives.
- NVIDIA NewsroomMore Performance From Every Byte: How Full-Stack Codesign Solves AI’s Memory Challenge
NVIDIA describes how its full-stack codesign approach addresses growing GPU memory pressure from larger models, longer contexts and agentic applications by designing GPUs, CPUs, networking, storage and software together across a memory hierarchy. Techniques include NVFP4 for model weights and KV cache, distributing weights across up to 72 GPUs in NVL72, offloading inactive context to CMX context memory storage via Dynamo, and keeping RAG indexes outside HBM with Nemotron Retriever and cuVS. NVIDIA reports that from Blackwell Ultra to Vera Rubin, HBM capacity per GPU rose roughly 3% while agentic throughput per dollar rose up to 30x on SemiAnalysis AgentX.
Activity for this story
- Current comparable activity
- 7
- Peak comparable activity
- 10Nov 6 01:00
- Change in the past 24 hours
- –
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.