Google explains cost optimization of node pools for AI agent sessions on GKE
Understand this story
Google has published a technical article explaining how to plan node pools for AI agent sessions on GKE to reduce the cost of idle sessions. The article notes that agent sessions stay open all day, but are actually working only about 22% of the time; a GitHub Copilot agent tracking study shows that it spends about 80% of its time waiting for the user. Google says Agent Substrate 1.0 is now generally available on GKE, and SuspendActor can snapshot idle actors to object storage and release workers while keeping the session open. However, Agent Substrate does not suspend idle actors on its own; the agent framework or its fronting service must call the suspend API after an idle timeout. The article also notes that when idle actors occupy all workers, requests for new actors wait up to 5 seconds by default and then fail with HTTP 503. The AX mentioned in the article is still in development and has not yet released its first stable version.
AI overview based on source reports · Updated 12 min ago
Coverage timeline
Follow the coverage to explore different perspectives.
- Google Developers BlogAgents are sessions, not requests: size GKE node pools for AI agent sessions
This article argues that AI coding agent sessions stay open all day but work only part of the time—about 22% of an 8-hour day at 2 prompts an hour—so sizing GKE node pools for open sessions overpays for idle time. It describes how Agent Substrate and AX park idle sandboxes in object storage and free their workers, with a cost model showing parking saves 40% of the node bill for 100 coding agents at 2 prompts an hour, while one Pod per session becomes cheaper above roughly 4.5 prompts an hour. It also notes that Agent Substrate does not suspend idle actors on its own, that the warm pool must be sized for the busiest hour, and that the pool should be scaled to zero after hours with a scheduled job.
Activity for this story
Not enough continuous observations to show a trend yet.