API Failover

Create or improve a lightweight failover layer for AI APIs.

Goals

Build systems that:

• - detect unavailable or degraded providers/models
• classify failures before retrying blindly
• switch to a safe fallback chain
• avoid hammering broken endpoints
• recover back to preferred providers after cooldown

Workflow

1. 1. Identify the call path.
2. Classify failure modes.
3. Define a fallback policy.
4. Add health memory.
5. Implement guarded retries.
6. Emit observable logs.
7. Validate with forced-failure tests.

Use the detailed rules below and the bundled scripts instead of re-inventing routing logic each time.

Practical defaults

Error classes

Use these normalized categories:

• - INLINECODE0
• INLINECODE1
• INLINECODE2
• INLINECODE3
• INLINECODE4
• INLINECODE5
• INLINECODE6
• INLINECODE7
• INLINECODE8

Suggested routing behavior

• - AUTH_ERROR, BAD_REQUEST: fail fast; do not retry other providers unless config explicitly maps to another credential set.
• INLINECODE11: short backoff, then fallback.
• INLINECODE12, SERVER_ERROR, NETWORK_ERROR, MODEL_UNAVAILABLE, UNKNOWN_TRANSIENT: retry briefly, then fallback.
• INLINECODE17: mark provider unavailable for a longer cooldown and fallback immediately.

Circuit breaker defaults

Start with:

• - open after 3 consecutive transient failures
• cooldown INLINECODE19
• half-open with 1 probe
• close after 1-2 successful probes

Configuration pattern

Keep policy in config, not hard-coded logic.

Recommended shape:

• - provider registry
• task profiles with ordered fallback chains
• retry policy
• circuit-breaker policy
• per-provider overrides

Design guidance

• - Prefer fewer, well-understood providers over large fallback chains.
• Keep the fallback chain semantically compatible when possible.
• Separate "best quality" from "must return something" behavior.
• Keep downgrade rules explicit; avoid silent huge capability drops for critical tasks.
• For tool-using agents, treat provider switching as a reliability event and report it when user-visible quality may change.

Semi-automatic deployment model

Use this skill to discover the environment, generate a production-ish config, run a local HTTP failover proxy, and verify health.

Do not claim full autonomous takeover unless the environment-specific integration is actually completed.

References

Read these only when needed:

• - references/config-example.yaml for a compact policy example
• INLINECODE23 for a more practical multi-provider template
• INLINECODE24 for a ready-to-edit production template
• INLINECODE25 for failure-injection and validation cases
• INLINECODE26 for local proxy deployment and environment-variable setup
• INLINECODE27 for a user-systemd service example

Bundled scripts

scripts/discover_env.py

Inspect the current environment.

scripts/generate_config.py

Generate a production-ish YAML config from simple defaults.

scripts/failover_proxy.py

Run a minimal CLI failover call path.

scripts/http_proxy.py

Expose a single local OpenAI-compatible entrypoint.

Endpoints:

• - INLINECODE32
• INLINECODE33

Optional request header:

• - INLINECODE34

scripts/selfcheck.py

Validate that the local proxy is reachable and can process a minimal chat request.

scripts/bootstrap_failover.py

Run the semi-automatic bootstrap flow:

• - discover environment
• generate config
• optionally start the proxy
• run self-check
• print next actions

Example:

CODEBLOCK0

Keep these scripts small and inspectable. Extend them instead of turning SKILL.md into code-heavy instructions.