Core Features

Fallback & Retry

Real model providers have bad minutes — a rate-limit spike, a brief overload, a timeout. Fallback & retry make a single request survive them automatically: VectorAxis re-tries a stumble, and if a provider is truly down it switches to a backup — all driven by a few request headers, with no code in your app to catch errors and loop. The caller just gets the first good answer.

Two ways to get this — headers vs. a routing config

This page covers the per-request approach: you set retry and fallback on each call with headers. It’s perfect for ad-hoc resilience and quick experiments. For the same behaviour as a reusable, centrally-managed policy — where the backup keys live in the platform instead of in every request — use a routing config with the FALLBACK strategy and a per-target retry.

Approach	Best when…
Headers (this page)	you want resilience on a specific call, or to vary it per request. Backup credentials travel in the request.
Routing config	you want one reusable policy across the app, with backup keys managed centrally and changeable without a redeploy.

How a request becomes resilient

The two features stack in a simple order:

1. The primary call runs against your main credentials (a virtual key, or a direct provider + key).
2. Retries repeat that same primary call when it fails with a retryable error, pausing a little longer between each attempt, up to your retry count.
3. The fallback chain kicks in only after the primary’s retries are exhausted: it tries each backup provider in order until one succeeds.
First success wins and is returned immediately. If everything fails, the last error is returned, so you still get a meaningful status code.

Streaming is not retried or failed over. A streamed response (stream: true) starts sending tokens immediately, so it can’t be safely rewound to another attempt. Retry and fallback apply to standard (non-streaming) requests.

Retry — ride out a blip

Many failures are momentary: a provider briefly rate-limits you, or hiccups for a second. A retry simply asks again, with a short, growing pause in between, and usually the second attempt just works — no failover, no error shown to your user. Retries are off by default; turn them on by setting a retry count.

retry headers

Header	What it does	Accepted values	Default
x-retry-count	How many times to retry the primary after the first attempt. 0 means no retries.	a whole number 0–5 (higher is capped to 5)	0
x-retry-status-codes	Which error codes are worth retrying. Replaces the default list.	comma-separated HTTP codes, e.g. 429,503	429, 500, 502, 503, 504, 529

What “back-off” means. Instead of hammering a struggling provider, VectorAxis waits a little longer before each retry — about 1s, then 2s, 4s, 8s, 16s — capped at 60 seconds. This “exponential back-off” gives a busy provider room to recover and is the industry-standard way to retry safely.

The provider can set the pace. If a provider returns a numeric Retry-After header (telling you exactly how long to wait), VectorAxis honours it instead of the back-off schedule — so you respect the provider’s own guidance.

Retry the right errors. The defaults — 429 (rate-limited), 500/502/503/504 (server errors & timeouts), and 529 (overloaded) — are the failures that often clear on a second try. A 400 (bad request) or 401 (bad key) won’t fix itself, so it’s left off the list and fails fast.

Fallback chain — switch providers on failure

When the primary is genuinely unavailable — even after retries — VectorAxis can hand the request to a backup. Define up to five backups; they’re tried in order until one succeeds. Each backup is a complete destination: its own provider, its own key, and (optionally) its own model — so you can fail over from, say, OpenAI to Anthropic to Groq, swapping the model to suit each one.

Number your backups starting at 1. The set is read in order and stops at the first gap — if x-fallback-2-* is missing, anything numbered 3 and up is ignored.

fallback headers (n = 1–5)

Header	What it does	Accepted values	Default
x-fallback-{n}-provider *	The backup provider to try for attempt n.	a provider id (e.g. anthropic, groq)	—
x-fallback-{n}-api-key *	The API key for that backup provider.	the provider’s key	—
x-fallback-{n}-model	Use a specific model on this backup. Handy when providers don’t share model names.	a model id for that provider	the request’s model

Retries first, then fallback. The chain only advances after the current destination’s retries are used up. So a request might retry the primary twice, give up, try backup 1 (with its own retries if you set them), and only then move to backup 2 — all transparently, returning the first success.

A note on keys in headers. Header-based fallback carries the backup API keys in the request. If you’d rather not put provider keys in every call, use a routing config: the backups are virtual keys stored in the platform, and the request only carries the config’s slug.

Putting it together

A primary virtual key, retried up to twice on rate-limits and timeouts, with two backups across different providers:

http

POST /chat/completions
Authorization: Bearer vk-1a2b3c4d5e6f7a8b
x-retry-count: 2
x-retry-status-codes: 429,503,504
x-fallback-1-provider: anthropic
x-fallback-1-api-key: sk-ant-...
x-fallback-1-model: claude-haiku-4-5
x-fallback-2-provider: groq
x-fallback-2-api-key: gsk_...
x-fallback-2-model: llama-3.3-70b-versatile

What the response tells you

Every response reports what actually happened, so you can monitor how often resilience kicks in:

Response header	Meaning
x-retry-count	How many retries were actually performed before a result was reached (0 if the first attempt succeeded).
x-fallback-provider	Which backup provider served the request, if the chain advanced. Absent when the primary handled it.
x-virtual-key-slug	The key that ultimately served the request.

These also flow into your request logs, so you can see retry and fallback rates over time and spot a provider that’s degrading before it becomes an outage.

Rule of thumb: add a small x-retry-count (1–2) to smooth over blips, and one or two fallback providers for real outages. Want it as a reusable policy instead of per-request headers? Build a routing config.