Cascade：面向编程智能体的本地 AI 网关，聚合 60 个免费模型

Cascade is a local AI gateway for coding agents: one OpenAI-compatible endpoint (http://localhost:3000/v1) in front of a cascade of 60 free coding models across 6+ providers with automatic failover — you never see a refusal while at least one model is alive. Plug it into opencode, Continue, Cline, Aider or any OpenAI-compatible client. A built-in VLESS-Reality relay routes only geo-blocked providers through the tunnel; everything else goes direct.

Русский

Table of contents

1. Features
2. Architecture
3. Quickstart
4. Connecting agents
5. Model IDs
6. VPN relay
7. Security
8. Testing & Contributing
9. Docs
10. Screenshots
11. FAQ
12. License

Features

• 60-model free cascade, 6+ providers — the active set (fast-coding) is ordered by provider priority and per-model score; the router walks the cascade until one model answers.
• Automatic failover — on HTTP 429 / quota exhaustion / 5xx / timeouts / invalid bodies. Content validation catches "HTTP 200 with an error payload" and treats it as a failure too.
• Circuit breakers — CLOSED / DEGRADED / OPEN / HALF_OPEN with escalating cooldowns, quota-pause that honors Retry-After, cached probe results, sticky auth-error marking, provider-skip, a lastResort model, and autoHeal.
• Honest routing — three explicit forms: cascade (auto-cascade), cascade:<set> (a named set), cascade:@<provider>/<model> (strict pin, failover off). Unknown ids → 404 model_not_found; a bare provider/model line is not a pin and is routed as a cascade request.
• Own router engine on :19080 — HTTP API, SSE streaming, Anthropic-compatible /v1/messages, v1/v2 config with atomic writes, persisted state (breakers, quotas, probes, history).
• Own model catalog — 530 entries with provenance (cascade-router/catalog.json, fields provider/model/tier/score/context/provenance/note), regenerated by scripts/catalog-bootstrap.mjs.
• Built-in VLESS-Reality relay — parses your subscription, egress-probes every node with parallel sing-box instances, chooses nodes probe-aware, validates candidates with a pre-flight sing-box check, rolls back to a known-good config, and uses a NO_PROXY policy so only geo-blocked providers go through the tunnel.
• Web dashboard (React + Vite + Tailwind) — catalog with filters, cascade panel (Auto/Manual), VPN manager, code sandbox.
• Security — everything binds to 127.0.0.1 (including the Vite dev server), provider keys live on the backend only (the UI sees just hasApiKey), statuses are masked.
• Operations — launchd KeepAlive templates in configs/launchd/ (labels com.cascade.*), regression runner scripts/regression.mjs (PASS / WARN / FAIL).

Architecture

agents (opencode / Continue / Cline / Aider / any OpenAI-compatible client)
  │ OpenAI-compatible HTTP → http://127.0.0.1:3000/v1
  ▼
facade :3000  (OpenAI-compatible API + dashboard)
  │
  ▼
router :19080  (cascade engine: 60 models / 6+ providers, failover)
  │
  ├── [providers] directly (NO_PROXY)
  └── sing-box :10808 (VLESS-Reality) → [providers] only geo-blocked

Stack: Bun 1.4.2 · TypeScript · Express · React · Vite · Tailwind · sing-box 1.14.1 (known defect: the xhttp transport is not supported by sing-box 1.14.1 — see FAQ).

Quickstart

Prerequisites:

• Bun 1.4.2+ (single runtime: install deps, run, build, test).
• sing-box — optional, only for the VLESS-Reality relay: brew install sing-box.
• Platforms: macOS / Linux / Windows — the core gateway works natively on all three (Bun ships official binaries); the launchd service templates (configs/launchd/) are macOS-only.

git clone https://github.com/MrTodone/cascade
cd cascade
bun install

# 1) Router config (required — the router will not start without it):
cp configs/router.config.example.json cascade-run/router/config.json
#    then replace every YOUR_<PROVIDER>_KEY in that file with a real key.
#    A provider without a key is simply inactive (hasApiKey = false).

# 2) Optional facade keys (sandbox / code generation):
cp .env.example .env
#    fill GEMINI_API_KEY / OPENROUTER_API_KEY / GROQ_API_KEY.

# 3) Run (dev, from package.json scripts; for a production bundle use
#    `bun run build` then `bun run start`):
bun run dev

# 4) Verify:
curl http://localhost:3000/v1/models

First-run without config: since v0.2.0 the facade shows the first-run wizard (GET /api/setup/status → GET /api/setup/apply); it writes cascade-run/router/config.json + .env from the values you paste, then restarts the router engine.

Prebuilt binaries (recommended, v0.3.0)

Self-contained downloads — no Bun/Node required (the Bun runtime, the router engine and sing-box are bundled). Grab them from the Releases page; verify with SHA256SUMS (sha256sum -c SHA256SUMS on Linux, shasum -a 256 -c SHA256SUMS on macOS).

Windows. Unpack the zip anywhere and double-click START-Cascade.cmd. On first launch Windows SmartScreen may show "Windows protected your PC" — click More info → Run anyway (the app is unsigned; it is the file you downloaded from this repository). The browser opens http://localhost:3000 with the first-run wizard: paste API keys for at least one provider and save. Details: README-WINDOWS.txt in the zip.

Linux.

tar xzf cascade-v0.3.0-linux-x64.tar.gz
cd cascade-v0.3.0-linux-x64
./START-cascade.sh          # or: ./cascade  (run from the unpacked directory)

For systemd autostart use the templates in configs/systemd/; full guide in README-LINUX.txt.

Tunnel manager (vpn.tunnelManager): auto (default) selects launchd on macOS and builtin on Windows/Linux; builtin spawns the bundled sing-box as a child of the facade; external only probes an already-running tunnel. The active manager is reported by GET /api/vpn/status → manager, and the dashboard Stop button (POST /api/app/shutdown) shuts down the router + tunnel and exits cleanly on every platform.

Point your agent at http://localhost:3000/v1 and you are done.

Connecting agents

opencode — add a local provider (see configs/opencode.example.json):

{
  "provider": {
    "cascade": {
      "npm": "@ai-sdk/openai-compatible",
      "options": { "baseURL": "http://localhost:3000/v1", "apiKey": "cascade-local" },
      "models": { "cascade": { "name": "Cascade auto-router" } }
    }
  },
  "model": "cascade/cascade"
}

Continue (~/.continue/config.json):

{
  "models": [{
    "title": "Cascade",
    "provider": "openai",
    "model": "cascade",
    "apiBase": "http://localhost:3000/v1",
    "apiKey": "local"
  }]
}

Cline — Settings → OpenAI-compatible provider: Base URL: http://localhost:3000/v1, API Key: anything, model cascade.

Aider:

aider --openai-api-base http://localhost:3000/v1 --openai-api-key dummy --model cascade/cascade

Any OpenAI-compatible client works the same way.

Model IDs

VPN relay

Purpose. Some free AI providers (e.g. Google AI Studio) are geo-blocked from certain regions. Cascade ships a built-in VLESS-Reality relay that routes only those geo-blocked providers through an outbound tunnel; every other provider stays direct. Browser, games and local services are untouched.

Bring-your-own subscription. The subscription URL is configured only in the config file (vpn.subscriptionUrl), never in code or examples. Empty/missing value = the relay is gracefully disabled: direct providers keep working and geo-blocked ones simply fall over through the cascade. When the URL is set, server/vpnService.ts re-reads the subscription hourly, egress-probes every node with parallel sing-box instances, and applies the best candidate.

• Config: cascade-run/router/config.json → vpn.subscriptionUrl (your own URL).
• For an existing macOS install that keeps its own service label: vpn.singboxServiceLabel. Empty = default com.cascade.singbox; on migration set the actual launchd service label (the public examples in configs/launchd/ use com.cascade.*).

Install sing-box:

brew install sing-box

launchd templates (configs/launchd/): com.cascade.server.plist.example (the facade) and com.cascade.singbox.plist.example (the tunnel). They use <BUN_BIN>, <CASCADE_ROOT>, <SINGBOX_BIN>, <SUBSCRIPTION_HOST> placeholders — resolve them for your machine.

OS support, honestly: macOS — full functionality (launchd KeepAlive + automatic tunnel steering). Linux (v0.3.0+) and Windows (v0.2.0+) — the core gateway runs natively from the prebuilt binaries, and the tunnel is managed by the builtin manager (the facade spawns the bundled sing-box as a child and steers traffic). Automatic service management: launchd templates for macOS (configs/launchd/) and systemd units for Linux (configs/systemd/); on Windows there is no service installer yet — start START-Cascade.cmd directly or wire it into Task Scheduler / NSSM. See the FAQ for the full matrix.

NO_PROXY policy: the following are direct — ipv4/ipv6 loopback, Google AI Studio, Cloudflare, OrcaRouter, Groq console, GitHub, HuggingFace, ollama.com, Mistral, LLM7, DashScope/Qwen, Z.ai. Traffic to OpenRouter and the Groq API host goes through the tunnel (:10808).

Known defect: sing-box 1.14.1 does not understand the xhttp transport — such nodes fail pre-flight and are skipped; a sing-box check exits FATAL. A fix is an upstream sing-box upgrade or filtering xhttp candidates (tracked, not fixed in v1).

Disclaimer. Free upstream providers are rate-limited by their own quotas; the cascade only hides refusals from the agent while at least one model answers. The relay is intended for legitimate geo-restrictions — respect provider Terms of Service and your local laws.

Security

• The facade binds 127.0.0.1 only (enforced in server.ts); the Vite dev middleware/HMR is also pinned to loopback. Reaching :3000 or :24678 over the LAN is refused.
• /v1 is localhost-only and, as shipped, has no auth — do not expose it beyond your machine.
• Provider keys live on the backend. The dashboard only receives hasApiKey booleans.
• Statuses and responses are masked: ips are truncated/masked, node uuid/pbk/sid/raw URLs are never returned.
• Honest HTTP 413 on bodies >10 MB.
• DNS-rebinding guard in the router (only 127.0.0.1, localhost, ::1 hosts accepted).

Testing & Contributing

Offline (no keys, no network, one machine):

bun run lint                       # tsc --noEmit
./scripts/cascade-router-test.sh   # mock suite: 113 PASS / 0 FAIL
bun cascade-router/tests/config.test.ts   # 15 PASS
bun cascade-router/tests/persist.test.ts  # 20 PASS
bun cascade-router/tests/heal.test.ts     # 19 PASS

Live regression (needs a running router and, for some checks, provider keys):

bun scripts/regression.mjs   # PASS / WARN / FAIL; external degradations → WARN, not FAIL

See CONTRIBUTING.md.

Docs

• OPERATIONS.md — ports, architecture, recovery runbook.
• CORE-SPEC.md — the clean-room engine spec (why there are no external router patches).

Screenshots

FAQ

Is this a fork? No. Cascade has its own architecture and its own engine. It started as a wrapper around the MIT-licensed free-coding-models package, then the mechanics were rewritten from scratch (clean-room, spec in docs/CORE-SPEC.md) and all external router packages/patches were removed. Two contribution PRs were merged upstream.

Do I need API keys? Only for the providers you want to use — they are free-tier models. A provider without a key is simply inactive.

Do I need Bun or Node? No — the prebuilt binaries (Windows zip, Linux tarball) are self-contained: the Bun runtime, the router engine and sing-box are bundled. You only need Bun to run from source.

Does it work without VPN / sing-box? Yes. Direct providers work immediately; geo-blocked ones gracefully fall over through the cascade. The relay starts only when vpn.subscriptionUrl is configured. On Windows the dashboard panels for VPN sync/pause/stop manage sing-box.exe as a child process of cascade.exe (no services installed); live egress on Windows is not exercised in CI — a failing subscription node disables the relay gracefully and direct providers keep working.

Which OS? macOS — full functionality, including launchd service templates (configs/launchd/). Linux (v0.3.0+) — prebuilt cascade-v0.3.0-linux-x64.tar.gz, builtin tunnel manager (bundled sing-box), systemd units in configs/systemd/. Windows (v0.2.0+) — prebuilt zip (cascade.exe + cascade-router.exe), bundled sing-box.exe, a first-run wizard in the dashboard, and START-Cascade.cmd, with the builtin tunnel manager. Service installation outside macOS is manual (systemd / Task Scheduler / NSSM). The project is developed and regression-tested on macOS; the Linux and Windows core gateways are expected to work, but the live regression suite is macOS-only.

Are my keys safe? They stay local, on a loopback-only backend; the dashboard never receives them.

What about rate limits? Free tiers rotate through the cascade automatically; the breaker/quota logic keeps retries bounded per request and per provider.

Why does it not start without config? The config file is mandatory for the router engine: cp configs/router.config.example.json cascade-run/router/config.json.