Model Coordination

Sometimes one model is not the best solution for every task.

Instead:

                Request
                   ↓
             Model Selection
            /       |       \
           ↓        ↓        ↓
       Model A   Model B   Model C
        Search    Coding   Summary

Each model can specialize in a capability.

Example:

Customer Request
       ↓
Classification Model
       ↓
 ┌─────┼──────┐
 ↓     ↓      ↓
Code  Search  Summary
FM     FM      FM

This can improve:

• performance
• cost
• task specialization

Multiple models can produce outputs.

             Input
               ↓
       ┌───────┼───────┐
       ↓       ↓       ↓
    Model A  Model B  Model C
       ↓       ↓       ↓
       └───────┼───────┘
               ↓
        Aggregation Logic
               ↓
          Final Output

Aggregation logic might select or combine outputs according to application requirements.

Do not automatically choose the largest model.

Evaluate:

Quality
+ Cost
+ Latency
+ Capability
+ Context requirements
+ Availability
+ Security
+ Workload requirements

Most powerful model ≠ automatically best model.

The correct model is the one that satisfies the application's requirements.