FastAPI + Ollama

LLM Failover Gateway

Route chat and generation requests through two local Ollama servers first, then fall back to an internal Dockerized phi4 service when the external hosts are unavailable.

Open API Docs Manage Upstreams

Routing Order

Server 1
Server 2
Internal phi4 fallback

Public API endpoints live under /api/v1/ and require your API key.

Chat + Generate

Expose stable endpoints for chat and prompt-based requests while abstracting upstream selection.

Configurable Backends

Adjust URLs, priority, timeouts, and model support from the admin page without changing code.

Live Status

See which Ollama hosts are healthy and which models are available by probing /api/tags.