Ask HN discussion on the speed of running frontier AI models locally
Understand this story
A developer asked on Hacker News: is there purchasable local hardware that can run frontier AI models at usable speeds, to offset cloud bills? In the discussion, someone pointed out that the fast local hardware inference promised by EXO labs eight months ago has not materialized, and getting a single response using Kimi K2.6 via hermes takes about a minute and a half. So far the discussion has not produced a hardware solution meeting this requirement, and the question remains at the stage of being asked.
AI overview based on source reports · Updated 22 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Hacker News AIAsk HN: Can anyone run frontier AI models at usable speeds?
An Ask HN post notes that eight months ago EXO labs promised fast inference on local hardware, but it is not fast enough: getting a response through hermes using Kimi K2.6 takes a minute and a half. The poster asks whether there is hardware a developer can buy that can actually offset cloud bills.
Activity for this story
Not enough continuous observations to show a trend yet.