Hacker News AI· anneouyang·· 2 days agoAI score58
Standard Kernel Used AI to Make Intel's Abandoned Gaudi2 Outperform Nvidia H100
Standard Kernel Used AI to Make Intel's Abandoned Gaudi2 Outperform Nvidia H100
AI summary
Standard Kernel reports that software optimization alone raised Qwen3.5-27B inference on a single Intel Gaudi2 to 3.21x cached-prefill and 2.84x decode throughput over its vLLM baseline, exceeding an NVIDIA H100 80GB PCIe by 7.92% and 60.35% respectively. At rental prices of $0.60/hour for Gaudi2 versus $3.50/hour for H100, the optimized Gaudi2 delivers 9.4x more decoded tokens per dollar. The post attributes Gaudi2's limited adoption partly to software support and describes Standard Kernel's system for autonomously experimenting with and benchmarking inference-stack changes.
Source: Hacker News AI · standardkernel.com