Reducing token costs and latency in Amazon Bedrock AgentCore JSON workloads
Reducing token costs and latency in Amazon Bedrock AgentCore JSON workloads
This AWS post describes three composable strategies used to build a Flightscape crew training schedule optimization agent on Amazon Bedrock AgentCore that handles 20 MB+ JSON payloads: offloading heavy artifacts to Amazon S3 and persisting only lightweight references in AgentCore memory, paging large result lists on demand, and compacting oversized dictionary outputs into descriptors. Across a 5-question conversation run, total token usage dropped by approximately 71% (from ~289,500 to ~82,900 tokens) and average trace latency fell nearly 80% (from 15.4 s to 3.1 s); across 100 sample questions, tokens fell from 23.6 M to 8.5 M and average latency from 7,930 ms to 2,461 ms. The heaviest single query saw input tokens drop 75%, output tokens 83%, and turn latency 94% (from ~165 s to 9.9 s).
Source: AWS Machine Learning Blog · aws.amazon.com