A fallback model list for one access key.
Model Failover lets an Owner add an ordered list of fallback models for one access key. Enabling the module creates no rule until you add one.
What it does
Open Gateway modules and enable Model Failover. Then open Gateway and add a fallback rule: choose one access key and list one to five models, separated by commas. The rule is stored for that key only. Removing the rule, or disabling the module, deletes the rules that belong to this workspace.
Response status
Failover triggers on HTTP 502, 503,
504, or a connection timeout. The Gateway tries each model
in order until one returns a successful response. If all models fail,
the last error is returned to the caller.
https://YOUR-SITE/v1/chat/completions
$ access key has failover: gpt-4o, gpt-4o-mini, claude-3-5-sonnet
The caller sees only the final response — no intermediate retries are
exposed. A header
X-Failover-Attempts reports how many models were tried.
What it does not do
- Create a fallback chain when you only enable the module.
- Change another workspace's rules.
- Retry a request whose outcome is unknown (e.g. a partial stream or an HTTP 200 with a content error).
- Retry streamed responses mid-stream.
- Apply to a key without an explicit fallback rule.
Status in the dashboard
Gateway → Gateway modules reports whether Model Failover is loaded. Each access key on the Gateway page shows its current failover model list, if any.
Limits
| Setting | Description |
|---|---|
| Models per rule | 1–5, tried in order |
| Triggers | HTTP 502, 503, 504, connection timeout |
| Rules per workspace | One per access key |
| Streamed responses | Not retried mid-stream |
| Monthly fee | $29 per workspace |
Per-key monthly budget and requests per minute are already set when you create an access key. Those do not require this module.