iCreatAI News
Skip to content
Trending storyUpdating

Bedrock open-weight models support explicit prompt caching

1 reports1 sources2 hr agoUpdated

Understand this story

AI overview

Open-weight models on Amazon Bedrock now support explicit prompt caching: the already-computed state of the same prompt prefix can be reused to reduce input costs and response latency. In the tutorial, AWS says cache reads are billed at a lower rate than standard input, and savings accumulate each time a prefix is reused; however, depending on the model, cache writes may be billed at a higher rate than standard input. Caching has two modes: implicit caching is automatic and best-effort; explicit caching marks breakpoints via prompt_cache_options. The tutorial demonstrates both modes with the Chat Completions API and explains how to read cache usage from each response. Model availability and prompt caching support vary by AWS Region, and AWS recommends checking the model card to confirm the Regions supported for the target model.

AI overview based on source reports · Updated 1 hr ago

Coverage timeline

Follow the coverage to explore different perspectives.

Nov 16
  1. AWS Machine Learning Blog
    Use prompt caching with open-weight models on Amazon Bedrock

    Amazon Bedrock now supports explicit prompt caching for open-weight models, alongside automatic implicit caching, letting applications reuse a matching prompt prefix across requests to reduce input costs and latency. The post explains both modes through the Chat Completions API, shows how to read cache usage fields from each response, and applies implicit caching to an OpenCode coding agent session. It closes with best practices such as placing static content first, meeting the model's minimum prefix length, and tracking cache reads and writes.

Activity for this story

Not enough continuous observations to show a trend yet.