Skip to content

AIP-C01 · Study Guide

← Domain 4: Operational Efficiency and Optimization for GenAI Applications

4.2.1 · Lesson 5 of 16

Latency Reduction & User Experience

Minimize Time-To-First-Token (TTFT) and total response latency with streaming, pre-computation, and parallelism.

WHAT

Minimize Time-To-First-Token (TTFT) and total response latency.

HOW

  • Response Streaming: Bedrock Converse Stream / InvokeModelWithResponseStream with WebSockets or SSE.
  • Pre-computation: Pre-generate embeddings or static answers for frequent queries (Glue / scheduled Lambda).
  • Parallel Requests: Parallel tool calls or vector queries via asyncio or Step Functions parallel branches.

Key points

  • TTFT optimization
  • ConverseStream / InvokeModelWithResponseStream
  • WebSockets / SSE
  • Pre-compute embeddings / FAQs
  • Parallel tool and retrieval calls

Exam focus

Perceived latency improves fastest by streaming tokens via WebSockets/SSE — not by queuing or Glacier storage.

View progress