Standard Kernel says AI-optimized Gaudi2 inference surpasses H100
Understand this story
Standard Kernel published a technical blog post saying that it used AI to optimize Qwen3.5-27B inference on Intel Gaudi2, achieving 60% higher decode throughput and 8% higher cache prefill throughput than NVIDIA H100 80GB PCIe on long-context benchmarks. The company says Gaudi2 rents for $0.60 per hour versus $3.50 for H100, putting the former's cost at only 17% of the latter's, with 9.4 times as many output tokens per dollar. In the comparison, Gaudi2 used its custom modifications, while H100 ran vLLM 0.32.0. All the above figures are as stated by Standard Kernel and have not been independently verified.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIStandard Kernel Used AI to Make Intel's Abandoned Gaudi2 Outperform Nvidia H100
Standard Kernel reports that software optimization alone raised Qwen3.5-27B inference on a single Intel Gaudi2 to 3.21x cached-prefill and 2.84x decode throughput over its vLLM baseline, exceeding an NVIDIA H100 80GB PCIe by 7.92% and 60.35% respectively. At rental prices of $0.60/hour for Gaudi2 versus $3.50/hour for H100, the optimized Gaudi2 delivers 9.4x more decoded tokens per dollar. The post attributes Gaudi2's limited adoption partly to software support and describes Standard Kernel's system for autonomously experimenting with and benchmarking inference-stack changes.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 19 02:00
- Change in the past 24 hours
- -50%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.