Open-source large model inference is getting cheaper, but personal deployment is getting harder
Understand this story
Open-weight AI models are becoming cheaper to run inference on, yet harder for individuals and small teams to deploy on their own. The reason lies in model architecture: DeepSeek-V3 has 671B total parameters with 37B activated per token, while Llama 4 Maverick has 400B total parameters with 17B activated. Under the MoE architecture, total parameters determine the storage overhead of weights; at one byte per parameter, a trillion-parameter model needs about 1TB just for weights. Meanwhile, the batching advantages of inference engines such as vLLM favor large-scale service providers, making deployment costs hard for individuals and small teams to bear.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAI Is Getting Better. Running It Yourself Is Getting Harder
The article argues that large Mixture-of-Experts models such as DeepSeek-V3 (671B total, 37B active parameters) and Llama 4 Maverick (400B total, 17B active) can be efficient to compute yet costly to host, since all weights must remain available even when only a subset is activated per token. It contends that batching and shared deployments favor large providers, so open weights may become centralized in practice as self-hosting costs rise, and it suggests shared colocated clusters and research into compression, distillation, and expert offloading as responses.
Activity for this story
- Current comparable activity
- 4
- Peak comparable activity
- 10Nov 5 03:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.