Quick Reference

Model selection
    ↓
Benchmark + capability + cost + latency + constraints

Dynamic switching
    ↓
Abstraction + configuration

Resilience
    ↓
Fallback + Cross-Region + circuit breaker + graceful degradation

Customization
    ↓
SageMaker AI + LoRA/adapters + Model Registry + lifecycle

S3
= source documents

Embeddings
= semantic representation

Vector store
= retrieval index

Metadata
= filtering + context + governance

Incremental sync
= freshness + lower processing cost

Chunking
→ controls retrieval units

Embedding
→ converts content into vectors

Vector search
→ semantic retrieval

Keyword search
→ exact-term retrieval

Hybrid search
→ keyword + vector

Reranking
→ improves ordering of retrieved candidates

Query expansion
→ adds useful search terms

Query decomposition
→ breaks complex requests into subqueries

Retrieval API / function / MCP
→ standardizes access

A complete architecture can look like:

                         ┌──────────────────────┐
                         │      End User        │
                         └──────────┬───────────┘
                                    │
                                    ▼
                         ┌──────────────────────┐
                         │ API Gateway / App    │
                         └──────────┬───────────┘
                                    │
                                    ▼
                         ┌──────────────────────┐
                         │ Application / Lambda │
                         └──────────┬───────────┘
                                    │
                ┌───────────────────┼───────────────────┐
                │                   │                   │
                ▼                   ▼                   ▼
         Prompt Management    Retrieval Layer      Conversation
                │                   │                   │
                ▼                   ▼                   ▼
          Guardrails          Query Handling        DynamoDB
                                    │
                         ┌──────────┴──────────┐
                         │                     │
                         ▼                     ▼
                    Vector Search        Keyword Search
                         │                     │
                         └──────────┬──────────┘
                                    ▼
                                Reranker
                                    │
                                    ▼
                              Context Assembly
                                    │
                                    ▼
                              Model Router
                                    │
                       ┌────────────┼────────────┐
                       ▼            ▼            ▼
                    FM A          FM B       Fallback
                       │
                       ▼
                 Final Response
                       │
              ┌────────┴─────────┐
              ▼                  ▼
         CloudWatch          CloudTrail

Data ingestion:

Enterprise Sources
       │
       ▼
      S3
       │
       ▼
Validation
       │
       ▼
Processing / Normalization
       │
       ▼
Chunking
       │
       ▼
Embeddings
       │
       ▼
Vector Store
       │
       └──────────────→ Retrieval

### Use Prompt Engineering when:

• behavior can be controlled through instructions
• knowledge changes frequently
• no model behavior adaptation is required

### Use RAG when:

• the model needs current/private knowledge
• source documents change
• answers must be grounded in enterprise information

### Use Fine-Tuning / Customization when:

• consistent domain-specific behavior is required
• prompt/RAG alone is insufficient
• there is suitable training data

### Remember

Need new knowledge?
→ RAG

Need different behavior/style/task pattern?
→ Prompting or customization

Need both?
→ RAG + customization may be appropriate

# Keyword Search vs Vector Search vs Hybrid Search

Best for:

• exact IDs
• names
• codes
• exact phrases

Best for:

• semantic similarity
• paraphrased questions
• conceptual matching

Best when:

• both exact terms and semantic meaning matter

Keyword + Vector
       ↓
   Candidate Set
       ↓
    Reranker
       ↓
   Best Context

# Chunking Decision Framework

Choose when:

• documents are relatively uniform
• implementation simplicity matters

Choose when:

• meaning boundaries are important

Choose when:

• documents have strong structure
• parent/child context matters

# Model Selection Decision Framework

Evaluate:

Capability
+ Quality
+ Latency
+ Cost
+ Context
+ Modality
+ Availability
+ Security
+ Regional Constraints

Never select a model from one metric alone.

# Resilience Decision Framework

When a model fails:

Can retry safely?
   │
   ├── Yes → controlled retry
   │
   └── No
       ↓
Can fail over?
   │
   ├── Yes → alternate model / region
   │
   └── No
       ↓
Graceful degradation

# Data Pipeline Decision Framework

Is data valid?
   ↓
No → Reject / Quarantine / Alert
Yes
   ↓
Does it require transformation?
   ↓
Yes → Process / Normalize
No
   ↓
Does model require a specific format?
   ↓
Yes → Model Adapter
No
   ↓
Invoke FM

# Prompt Governance Decision Framework

Create
 ↓
Version
 ↓
Evaluate
 ↓
Approve
 ↓
Deploy
 ↓
Monitor
 ↓
Audit
 ↓
Improve / Rollback

# Common Domain 1 Mistakes

Better approach:

Choose the model that meets the actual requirements for quality, latency, cost, modality, and availability.

Changing enterprise knowledge is usually a retrieval/data problem rather than a reason to continuously fine-tune the model.

The vector store is generally a retrieval index over source information.

Exact identifiers and terminology may require keyword retrieval.

Metadata enables filtering by:

• date
• department
• region
• document type
• permissions
• classification

More context is not automatically better.

Relevant context should be selected and managed.

Use appropriate governance and safety mechanisms such as Guardrails where required.

Uncontrolled retries can increase load during outages.

Incremental updates are often more efficient.

Prompt changes need regression testing and measurable evaluation.

# Domain 1 Exam Traps

"Highest benchmark score" = best model

Not necessarily.

Evaluate against the actual business workload.

"RAG is always required for GenAI"

No.

Use it when external/current/private knowledge must be retrieved.

"Fine-tuning solves changing knowledge"

Usually not the first choice for frequently changing knowledge.

"Vector search is always better than keyword search"

No.

Hybrid retrieval can be better when exact terms matter.

"More chunks retrieved = better answer"

Not necessarily.

Too much irrelevant context can reduce quality and increase cost.

"More conversation history = better context"

Not necessarily.

Relevant context matters more than raw history size.

"Retry solves availability"

Not by itself.

Use controlled retries plus appropriate failover/degradation.

"Prompt instructions are security controls"

Not automatically.

Use appropriate platform security and safety controls.

"Vector database contains the authoritative document"

Usually the source repository remains authoritative.

"Complex architecture is better architecture"

No.

Use the simplest architecture that satisfies requirements.

Requirements
→ Constraints
→ Architecture
→ POC
→ Production

Think:

• business value
• latency
• cost
• security
• availability
• scalability

Evaluate
→ Select
→ Abstract
→ Route
→ Failover
→ Customize
→ Lifecycle

Key AWS:

• Bedrock
• SageMaker AI
• Lambda
• API Gateway
• AppConfig
• Step Functions
• Model Registry

Validate
→ Process
→ Normalize
→ Format
→ Enhance
→ FM

Key AWS:

• Glue Data Quality
• SageMaker Data Wrangler / Processing
• Lambda
• Transcribe
• Comprehend
• Bedrock

Source
→ Chunk
→ Embed
→ Store
→ Metadata
→ Maintain

Key AWS:

• S3
• Bedrock Knowledge Bases
• OpenSearch
• Aurora / RDS
• DynamoDB

Query
→ Transform
→ Search
→ Hybrid
→ Rerank
→ Context
→ FM

Key concepts:

• chunking
• embeddings
• vector search
• keyword search
• hybrid search
• reranking
• query expansion
• query decomposition
• retrieval APIs / function calling / MCP

Instruction
→ Context
→ Template
→ Guardrail
→ Evaluate
→ Version
→ Monitor
→ Improve

Key AWS:

• Bedrock Prompt Management
• Bedrock Guardrails
• Bedrock Prompt Flows
• Lambda
• Step Functions
• DynamoDB
• S3
• CloudTrail
• CloudWatch

2.1 AGENTS
│
├── Memory / State
├── Structured workflows
├── Safety controls
├── Model coordination
├── Human-in-the-loop
├── Tool integration
└── MCP

Agent behavior → Strands
Multiple agents → Agent Squad
Agent ↔ Tools → MCP

What does the application need?
          ↓
 ┌────────┼───────────┐
 ↓        ↓           ↓
On-demand Predictable  Custom
 ↓        ↓           ↓
Lambda   Bedrock      SageMaker
         Provisioned

• Evaluate latency, cost, traffic, model size, memory, GPU, token throughput, availability
• LLMs add memory, GPU, token processing, and model-loading concerns
• Use cascading: small FM for simple work, large/specialized FM for complex work

2.4 FM APIs
│
├── Sync
├── Async
├── Streaming
├── Retry
├── Rate limiting
├── Fallback
├── X-Ray
└── Model routing

2.5 APPLICATIONS
│
├── FM API interfaces
├── Amplify
├── OpenAPI
├── Prompt Flows
├── Business systems
├── Q Developer
├── Advanced agents
└── Troubleshooting

CloudWatch → logs/monitoring
X-Ray → distributed tracing

Step Functions = "What happens next?"
Lambda = "Do this operation."

Agent wants to act
       ↓
Does it have permission?
       ↓
      IAM
       ↓
Is input valid?
       ↓
Validation
       ↓
Can operation fail?
       ↓
Timeout / Retry / Circuit Breaker
       ↓
Should human approve?
       ↓
Human-in-the-loop
       ↓
Execute
       ↓
Stopping condition

Agent → Tools → Workflow → Security → Deployment → Enterprise Integration → FM API → Application → Monitoring

Agent behavior?          → Strands / Agent Squad
Tool interaction?        → MCP
Workflow/state?           → Step Functions
Execution?               → Lambda
Complex container tool?   → ECS
Model access?             → Bedrock
Custom model deployment? → SageMaker AI
API?                     → API Gateway
Async?                   → SQS
Events?                  → EventBridge
Permissions?             → IAM
On-prem?                 → Outposts
Edge/5G?                 → Wavelength
CI/CD?                   → CodePipeline / CodeBuild
Streaming?               → Bedrock streaming / WebSockets / SSE
Resilience?              → Backoff / rate limiting / fallback
Tracing?                 → X-Ray
Logs?                    → CloudWatch
Developer assistance?    → Amazon Q Developer

• Largest model ≠ always best — evaluate quality, cost, latency, capability, requirements
• MCP is not the agent — Agent → MCP → Tool
• Lambda executes; Step Functions orchestrates workflow/state
• Circuit breaker prevents repeated calls; it does not mean endless retries
• IAM is permissions, not model reasoning
• SQS is primarily asynchronous processing
• CloudWatch = logs/monitoring; X-Ray = distributed tracing
• Human-in-the-loop can be intentional design, not only failure recovery
• Streaming is optional — use when incremental output improves UX
• LLM deployment differs from traditional ML: memory, GPU, tokens, model loading

Need Runtime Content / Topic Safety? ──→ Amazon Bedrock Guardrails
Need to Find Sensitive Data in S3?  ──→ Amazon Macie
Need Continuous Endpoint Bias Tracking? ──→ SageMaker Clarify
Need Private Network Traversal? ───────→ AWS PrivateLink (Interface VPC Endpoints)
Need Model Compliance Documentation? ──→ SageMaker Model Cards
Need Full Request Payload Auditing? ───→ Bedrock Model Invocation Logging

User Request
    ↓
Gate 1 (Input Guardrails)
    ↓
FM Inference
    ↓
Gate 2 (Output Guardrails)
    ↓
Client

• Content filters: hate, sexual, violence, misconduct, insults, prompt attacks
• Denied Topics for natural-language policy bans
• Contextual Grounding = candidate + source reference chunks
• Separate system role from user role; enable Prompt Attack filters

• Explainability → Bedrock Agent Tracing + source attribution
• Fairness → Bedrock Model Evaluation / SageMaker Clarify
• Consequential decisions → human oversight + escalation on interventions

Static Prompt > 1,000 Tokens Repeated Frequently? ──→ Enable Bedrock Prompt Caching
Non-Realtime Asynchronous Ingestion Job? ─────────→ Use Bedrock Batch Inference (50% Off)
High Latency / Slow Response Perception? ────────→ Enable Response Streaming (TTFT Optimization)
Simple vs. Complex Query Mix Driving High Spend? ─→ Implement Model Cascading (Tiered Routing)
Need Token-Level Forensic Audit & Cost Tracking? ─→ Enable Bedrock Model Invocation Logging
High-Volume Sustained Traffic with Strict SLA? ────→ Bedrock Provisioned Throughput

• Compress prompts + bound maxTokens
• Cascade: cheap model first, escalate complex work
• Batch Inference for overnight/async (~50% off)
• Prompt Caching for static prefixes; semantic cache for FAQ repeats

• Metrics: InputTokenCount, OutputTokenCount, InvocationLatency, ThrottlingException
• Invocation Logging → CloudWatch/S3 for token forensics
• Alarms + SNS on token budgets and regressions
• Dashboards for cascade mix and cache hit rate

Output Cut Off Mid-Sentence? ─────────────→ Raise maxTokens (Output parameter)
Pre-Generation Token Validation Exception? → Chunk / Compress Input Payload
Exact Serial Numbers / Codes Missing in RAG? → Implement Hybrid Search (BM25 + Vector)
LLM Judge Prefers First Candidate? ───────→ Apply Pairwise Position Swapping
Agent Fails Tool Invocation with 403? ────→ Check IAM lambda:InvokeFunction Permissions
Newly Uploaded S3 Files Missing in RAG? ──→ Run StartIngestionJob Sync

• ROUGE/BLEU = lexical; BERTScore = semantic; Faithfulness = groundedness via LLM-as-a-Judge
• RAG eval stages: context relevance → groundedness → answer relevance
• Bedrock Model Evaluation: Automated or Human — assesses, does not auto-fix prompts
• Canary 5–10% traffic; quality gates block low faithfulness / truncation spikes
• Position swap A/B and B/A to neutralize judge bias