Resilient FM Systems

AI APIs can fail.

Possible failures:

• timeout
• throttling
• service unavailable
• network errors
• model errors
• dependency failure

A resilient architecture should handle them.

Instead of:

Fail → immediately retry
Fail → immediately retry
Fail → immediately retry

use increasing delays:

Attempt 1 → fail
   ↓
wait
   ↓
Attempt 2 → fail
   ↓
wait longer
   ↓
Attempt 3

The AWS SDK can support retry strategies including exponential backoff.

Protect the API/model from excessive traffic.

1000 requests
      ↓
Rate Limiter
      ↓
Allowed requests
      ↓
FM

API Gateway can participate in traffic management/rate-limiting patterns.

If the primary model fails:

Primary FM
    ↓
Failure
    ↓
Fallback FM
    ↓
Response

Or:

Primary AI unavailable
        ↓
Graceful degradation
        ↓
Reduced functionality

AWS X-Ray can help trace requests across service boundaries.

Example:

Client
 ↓
API Gateway
 ↓
Lambda
 ↓
Bedrock
 ↓
Database

X-Ray helps identify where latency/failure occurs.