4.3.5 · Lesson 15 of 16
Build Dashboards for GenAI Operations
Build CloudWatch dashboards that show tokens, latency, throttling, cache effectiveness, and model-routing mix.
WHAT
Build dashboards for day-to-day GenAI operations and FinOps review.
HOW
- Panels for Input/Output tokens and estimated cost.
- InvocationLatency and throttling.
- Cascade routing counts (cheap vs expensive models).
- Prompt cache / semantic cache hit rates.
Key points
- Ops dashboards
- Token and cost panels
- Latency percentiles
- Model cascade mix
- Cache hit rate
Exam focus
Operational dashboards should show token/cost, latency, throttles, and routing/cache effectiveness — not only uptime.