Model fallback

LLM fallback · model failover

Model fallback is a mechanism where an agent automatically switches to another model when the primary one fails, is overloaded or exceeds a limit, so the task is not interrupted.

Model fallback is a resilience policy: if the primary model returns an error, takes too long or exhausts its quota, the request is resent to an alternative model defined in advance. It is usually combined with retries with growing back-off and an attempt cap to avoid loops. It carries a cost to watch: the fallback model may be pricier or give different results, so it must be measured and capped like any other part of the agent. It is one of the error-handling practices for running agents in production.

Red Hat · 2026-10-02 · E3