5.2.5 · Lesson 14 of 14
Maintain Prompts After Production Drift
Maintain prompts and eval gates as production traffic, content, and model behavior drift over time.
WHAT
Maintain prompts after production drift using evaluation, canaries, and controlled promotion.
HOW
- Re-run fixed eval sets when Knowledge Bases or models change.
- Canary new prompts; watch faithfulness, truncation, and user feedback.
- Promote only through quality gates with rollback paths.
Key points
- Regression suites after KB updates
- Canary prompt rollouts
- Monitor faithfulness drift
- Update rubrics and datasets
- Controlled prompt promotion
Exam focus
Prompt maintenance is continuous — re-evaluate after content/model drift; do not change production prompts without gates.