Multi-Model LLM Architecture: When One Model Isn't the Right Answer (and the Operational Cost of Running Several)

A team ships on GPT-4. The next quarter a better model arrives, and someone adds it for the new use case rather than migrating.