FM API Interfaces

A GenAI API has unique requirements compared with traditional APIs.

Important considerations:

• streaming
• token limits
• timeouts
• retries
• request validation
• response handling

Models have limits around how much input/output they can process.

Conceptually:

System Prompt
+
Conversation
+
Retrieved Context
+
User Input
+
Output
=
Token Usage

Therefore applications should manage context carefully.

AI APIs can timeout.

Request
 ↓
Timeout
 ↓
Retry
 ↓
Success

But don't retry indefinitely.

Use controlled retries with appropriate limits.

A common architecture:

Frontend
   ↓
API Gateway
   ↓
Lambda / Service
   ↓
Bedrock
   ↓
FM

API Gateway can provide the application-facing API layer.