One endpoint, any client. This app now speaks the OpenAI API at https://llm-switchboard.icompaas.com/v1 — point Cursor, Cline, Codex CLI, Continue, or any OpenAI SDK at it, set the model to auto, and every request is router-picked across Groq / NVIDIA / Cloudflare with quota-aware auto-fallback (rate-limited or dead models go on cooldown and the next-ranked candidate answers). Ability inspired by OmniRoute — The Free AI Gateway ↗, which our own Trending feed surfaced.
Try it in 10 seconds — your provider key doubles as the OpenAI key (gsk_… = Groq, nvapi-… = NVIDIA, auto-detected)
curl
curl https://llm-switchboard.icompaas.com/v1/chat/completions \
-H "Authorization: Bearer $GROQ_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "auto", "messages": [{"role": "user", "content": "Why is the sky blue?"}]}'The auto models — zero-config routing, 40 live models behind them
| model id | what the router optimizes |
|---|---|
| auto | best overall model for the job (router classifies your last message) |
| auto/coding | best for code — routes to the top coding model |
| auto/fast | lowest latency (Groq LPU models win here) |
| auto/cheap | cheapest model that clears the quality floor |
| auto/smart | maximum quality, price be damned |
| groq:… / nvidia:… / bare id | any explicit catalog model (GET /v1/models) — router-ranked fallbacks still apply if it fails |
Hook up your tools
▮ Cursor
Settings → Models → OpenAI API Key: paste your Groq/NVIDIA key, then set Override OpenAI Base URL to https://llm-switchboard.icompaas.com/v1. Add auto (or auto/coding) as a custom model name.
base URL
https://llm-switchboard.icompaas.com/v1⬡ Cline / Roo
Provider: OpenAI Compatible · Base URL: https://llm-switchboard.icompaas.com/v1 · API key: your gsk_… / nvapi-… · Model: auto/coding.
model
auto/coding⌨ Codex CLI
In ~/.codex/config.toml add a provider with base_url below and env_key pointing at your provider key, then codex --model auto.
config.toml
[model_providers.switchboard]
name = "LLM Switchboard"
base_url = "https://llm-switchboard.icompaas.com/v1"
env_key = "GROQ_API_KEY"➤ Continue.dev
In config.json add a model with provider: "openai", apiBase below, your key, and model: "auto".
config.json
{"title": "Switchboard auto", "provider": "openai", "model": "auto", "apiBase": "https://llm-switchboard.icompaas.com/v1", "apiKey": "gsk_..."}🐍 OpenAI SDK · Python
Streaming, tools/function-calling and JSON mode all pass through untouched.
python
from openai import OpenAI
client = OpenAI(base_url="https://llm-switchboard.icompaas.com/v1", api_key="gsk_...") # your Groq key IS the api_key
r = client.chat.completions.create(model="auto/coding", messages=[{"role": "user", "content": "Fix this test"}])⚡ OpenAI SDK · JS/TS
Same idea in Node/TS — works from edge functions too (it’s just fetch).
node
import OpenAI from "openai";
const client = new OpenAI({ baseURL: "https://llm-switchboard.icompaas.com/v1", apiKey: "nvapi-..." }); // or gsk_...
const r = await client.chat.completions.create({ model: "auto", stream: true, messages: [...] });Bring any provider — address it as provider:model, the way Andrew Ng's aisuite does
🤗 Hugging Face — the meta-router inside the gateway
hugging face
curl https://llm-switchboard.icompaas.com/v1/chat/completions -H "Authorization: Bearer $HF_TOKEN" -H "Content-Type: application/json" -d '{"model": "hf:openai/gpt-oss-120b:cheapest", "messages": [{"role": "user", "content": "hi"}]}'Same provider:model convention for every onboardable provider: together:…, fireworks:…, cerebras:…, openrouter:… and more. Onboard one on Connections →
Compatible with aisuite (Andrew Ng's unified client, MIT · 15k★). It uses the identical provider:model convention, so its slugs work here drop-in — huggingface:… is accepted as an alias for hf:…. And because aisuite's OpenAI provider honors a custom base URL, you can point aisuite at this gateway (base_url = <origin>/v1) and let it handle routing, fallback and free-tier headroom for you.
How the fallback works — quota-aware, OmniRoute-style, in ~120 lines of zero-dep Node
1. The router ranks candidates for your request across every provider you have a key for (task classified from your last message, policy from the auto alias).
2. The gateway tries them in order — 429 puts the model on cooldown honoring Retry-After, 5xx cools it briefly, 404/410 (decommissioned) benches it for 6 h, 401/403 skips that provider for the request.
3. First healthy candidate answers. Streaming is raw SSE passthrough; the winner is reported in X-Switchboard-Model / X-Switchboard-Fallbacks response headers.
4. Cooled-down models are retried last rather than never — a fully-cooled list still gets one attempt instead of failing your request.
2. The gateway tries them in order — 429 puts the model on cooldown honoring Retry-After, 5xx cools it briefly, 404/410 (decommissioned) benches it for 6 h, 401/403 skips that provider for the request.
3. First healthy candidate answers. Streaming is raw SSE passthrough; the winner is reported in X-Switchboard-Model / X-Switchboard-Fallbacks response headers.
4. Cooled-down models are retried last rather than never — a fully-cooled list still gets one attempt instead of failing your request.
Loading live gateway status…
Endpoints
POST /v1/chat/completions — OpenAI-compatible chat (streaming, tools, JSON mode pass through)
GET /v1/models — the auto aliases + all 40 callable catalog models
GET /api/gateway/status — live provider keys, aliases, and active cooldowns
GET /v1/models — the auto aliases + all 40 callable catalog models
GET /api/gateway/status — live provider keys, aliases, and active cooldowns
What OmniRoute itself adds beyond this. The full OmniRoute ↗ project also does token compression (RTK/Caveman, 15–95% savings), 278 providers including subscription accounts, 18 routing strategies, an MCP server, and team quota-sharing. This gateway ports its core ability — one endpoint + smart fallback — in Switchboard's zero-dependency idiom. Roadmap: an Anthropic-native /v1/messages translation so Claude Code can point here directly.