Intelligent Caching Strategies

Bypass or cheapen model invocations with Bedrock Prompt Caching and semantic caching for repeated or similar queries.

WHAT

Maximize efficiency by caching static prompt prefixes and semantically similar Q&A pairs.

HOW

Prompt Caching (Bedrock): Cache pre-computed KV states for long static segments (system prompts, policy docs). Reuse cuts input token charges (up to ~90%) and latency.
Semantic Caching: Store response pairs in OpenSearch or ElastiCache; if similarity > threshold (e.g., 0.95), return cached answer without invoking Bedrock.

Static policy block > 1,000 tokens
        ↓
Prompt cachePoint (ephemeral)
        ↓
Subsequent calls reuse KV state

EXAM TRAP

Provisioned Throughput helps sustained capacity/SLA — it is not the direct answer for cutting repeated static preamble token cost (that is Prompt Caching).

Key points

Bedrock Prompt Caching
KV activation reuse for static prompts
Up to ~90% input token savings
Semantic caching in OpenSearch/ElastiCache
Similarity threshold bypass

Exam focus

Static prompts > 1,000 tokens reused frequently → Prompt Caching. Semantically identical FAQs → semantic cache hit without Bedrock.

View progress