AWS releases guide on four inference cost settings for Bedrock
Understand this story
AWS has released a practical guide explaining four request-level settings for Amazon Bedrock—cache duration, service tier, inference strength, and maxTokens—that can change token throughput and billing rates without modifying the model or prompt. The guide states that supported models also offer implicit prompt caching, which can attempt to reuse eligible prompt prefixes without setting cache control in the request; sending Flex requests to unsupported models returns a ValidationException rather than silently running as Standard. AWS says that at the time of writing, the published Flex rate for Nova 2 Lite in us-west-2 is half the Standard rate, and its billing reflects this. The guide shows how AWS tests each setting and how to verify their effects from response metadata, logs, and billing data.
AI overview based on source reports · Updated 2 days ago
Coverage timeline
Follow the coverage to explore different perspectives.
- AWS Machine Learning BlogFour Amazon Bedrock inference choices that affect cost
The post explains how four request-level Amazon Bedrock settings—prompt cache duration, service tier, reasoning effort, and maxTokens—can change token processing and billing rates without altering the model or prompt. It reports tests showing that a 1-hour cache pays off from the second request per hour when gaps exceed 5 minutes, that Flex on Nova 2 Lite processed the same tokens at half the Standard rate in us-west-2, that lowering effort from high to low cut median output tokens by 169% on an agent test, and that a 155-token maxTokens fence avoided 1,893 output tokens in one test. It recommends confirming model support, changing one setting at a time, and verifying effects through response metadata, CloudWatch metrics, invocation logs, and billing data.
Activity for this story
- Current comparable activity
- 3
- Peak comparable activity
- 10Nov 13 06:00
- Change in the past 24 hours
- -51%
Trends compare the same fully observed participants, which may cover fewer participants than current activity. Move the pointer or tap the chart to inspect hourly activity; use the left and right arrow keys with a keyboard.