3.4.2 · Lesson 14 of 15
Fairness Evaluations & Unbiased Outputs
Evaluate foundation models for demographic bias and fairness against baseline datasets before and after deployment.
WHAT
Evaluate foundation models for demographic bias and fairness.
HOW
- Run evaluations with Amazon Bedrock Model Evaluation or SageMaker Clarify.
- Compare against baseline datasets and track drift continuously in production.
Common mistake
Skipping fairness evaluation because quality demos look good on a narrow, non-representative sample.
Key points
- Bedrock Model Evaluation
- SageMaker Clarify fairness evaluations
- Baseline datasets
- Bias metrics
- Systematic evaluation before rollout
Exam focus
Execute systematic bias and fairness evaluations using Amazon Bedrock Model Evaluation or SageMaker Clarify against baseline datasets.