Sanctum Proxy

Every LLM request in the haus passes through one door. Every agent, every Claude Code session, every voice query funnels into a single ~10MB Rust binary on port 4040 — bouncer, translator, and accountant in one fast organism. It stands between your agents and the cloud providers who bill by the breath, and it does not negotiate.

It replaced a Python LiteLLM stack now retired from the live path — daemon, launch wrapper, and config all moved over in March 2026. (A docstring or two still says “LiteLLM” in the model-scout helper; the comment predates the rewrite.) The Rust version does the same job with less RAM, less latency, and no opinions about garbage collection. It knows 28 models across three providers — Gemini, OpenRouter, and local seats on Apple Silicon — keeps a fallback chain for nearly every one, and runs a budget that cuts an agent off mid-sentence if the math says so. It does not feel bad.

What It Does

The proxy receives an OpenAI-compatible request, decides which provider handles it, transforms it into that provider’s format, streams the response back, and tracks every token spent. One process — a single Rust daemon, no microservice archipelago.

It is config-driven. One config.yaml declares the models, fallback chains, quality tiers, and budget limits. Edit the file and the binary notices: it watches ~/.sanctum/sanctum-proxy/config.yaml and hot-reloads on change, logging Config reloaded successfully. No database — just a YAML file and convictions.

Performance Benchmark

The proxy is built for absolute minimum overhead. In a local ApacheBench stress test (ab -n 50000 -c 100), the core HTTP event loop turned in numbers that are almost rude:

Metric	Result
Throughput	142,000+ requests/sec (mean)
Mean latency	0.7ms (sub-millisecond)
P99 latency	2ms
Drop rate	zero — 50,000 requests, 100 concurrent connections, no failures

The routing layer adds virtually zero measurable overhead; your bottleneck is the speed of light to the cloud providers or the inference speed of your local models.

The 6-Layer Routing Engine

When a request arrives it falls through six layers of increasingly opinionated decision-making. The first layer with a strong opinion wins; the rest shut up.

🔍 Hover to zoom

Layer 1 is the velvet rope. claude-opus-max and council-secure are tier 0 — they go where they’re told, always, no second-guessing. If the agent asked for Opus, Opus is what it gets. Tout sauf de l’ostie de root beer.

Layer 6 is the bouncer, and here the metaphor is nearly literal: it is the same job Tommy worked at the front gate for fifteen years. When an agent burns through its daily token budget, every subsequent request is rerouted to a local model on Apple Silicon three feet away. The agent can still think, just smaller.

Budget System

2026-07-15: Claude CLI spend is now driven by hot-reloaded budget.yaml ($50/day shared USD pool). See that page for the green/red banners and local DEGRADED context fit.

Five AI agents share one wallet. Each gets a token bucket backed by a lock-free AtomicI64 — no mutexes, no contention, correct under concurrent burn because atomic operations have no feelings about fairness — the ideal temperament for handling money.

Claude Max free-token allocation is tracked separately, because free tokens are the most dangerous kind — they feel infinite until they aren’t. They get spent first, then paid tokens; when those hit the daily ceiling the circuit breaker fires and all traffic diverts to local models. Every transaction writes to a JSONL log via an async mpsc channel — non-blocking, batch-flushed.

Tip

Budget defaults to 500K tokens per agent per day. Claude Max contributes ~500K free daily tokens across all agents. Configurable numbers; non-optional math.

Subscription Zero-Rating

When an agent routes through a provider backed by an unlimited subscription — Anthropic (Claude Max) or Gemini (Studio Ultra) — the proxy applies a zero-rating policy: those requests draw nothing from the daily token budget and add nothing to USD spend tracking. They are genuinely free — local use of the subscription is effectively unlimited. Only metered calls — OpenRouter — draw down the limits.

The Claude Code Incident

The proxy was built for agents. Then Claude Code showed up — a different species of client. It broke the proxy in two ways that 52 passing tests never caught. The tests were green; the system was on fire.

Claude Code sends model IDs like claude-haiku-4-5-20251001. The proxy knew about 17 models; this was not one: unknown model, door closed. Worse, Claude Code authenticates with an OAuth token in the authorization header — and the proxy, in an act of breathtaking helpfulness, replaced it with its own. The request reached Anthropic carrying credentials from a different billing context, and was declined.

The lesson that launched 7 tests

When you expand a system’s scope — new client, new auth, new model format — the existing suite covers the old scope and says nothing about the new one. Write the new tests before deploying. The 52 that passed were correct. They were also irrelevant.

The fix: auto-passthrough for any claude-* model not in the config (no routing, no transformation, hands off), and a hard auth priority — the client’s OAuth token wins over the proxy API key, always. Seven new integration tests send requests the way Claude Code actually sends them. That reflex later became the foundation of the claude terminal’s online/offline lane — native Max when the uplink is up, the local Cathedral when it isn’t; the field note The Claude Code Lane tells the rest.

Streaming Token Tracking

This bug earned its own council session. Five AI agents argued streaming-protocol semantics, then voted unanimously to deploy the fix immediately rather than shadow-test. When the robots skip their own safety protocol, pay attention.

Streaming SSE responses — the kind Claude Code sends for every interaction — were returning zero tokens. The proxy faithfully logged input_tokens: 0, output_tokens: 0 for the most expensive traffic in the system, and the budget circuit breaker, whose sole job is to notice money being spent, reported all clear. The council’s verdict: “This is not a tracking bug. It is a budget circuit breaker that does not fire.”

The fix is StreamUsageExtractor, a state machine that reads token counts from an SSE stream’s final events — two-phase for Anthropic (input tokens in message_start, output in message_delta), single-phase for OpenAI-format providers with stream_options: {"include_usage": true} injected. If the connection drops first, it estimates from content bytes and marks the entry source: "estimated" — an honest guess beats a confident zero. Hard cap: 50 concurrent tracked streams; past that they get estimated, not buffered. Even accounting has limits.

Automated Model Discovery (Model Scout)

Static YAML does not update itself, so a model-scout service does the reading. Every Monday at 06:23 (via com.sanctum.model-scout.plist) it wakes on the Mac, pings the OpenRouter and Google AI APIs, and scores every model you aren’t using against the current config.yaml — points for low input/output token costs, bigger context windows, and recent release dates. When something interesting turns up (a new zero-cost tier, a capable $2/M context giant) it drops a structured Markdown digest — score breakdowns, cost metrics — into ~/.sanctum/memory/events, and fires a normal-priority request through the council-router to Qui-Gon, the infrastructure agent on the VM, who convenes the Jedi Council to vote on structural upgrades. The proxy does not upgrade itself. It nominates.

Configuration

The proxy reads a single config.yaml: a host/port pair, models as a list (each with a provider, an api_model, and an optional api_base for the local seats), a fallbacks map, and a budget block. There is no providers section — the provider lives on each model.

# ~/.sanctum/sanctum-proxy/config.yaml (abbreviated)
host: 0.0.0.0
port: 4040

models:
  - name: claude-opus-max
    provider: local                 # via the Claude Max bridge on :3456
    api_model: claude-opus-4        # unpinned alias — Max resolves it to the latest Opus (4.8 today)
    api_base: http://127.0.0.1:3456
    quality_tier: 0                  # never reroute
  - name: council-brain
    provider: local
    api_model: claude-opus-4
    api_base: http://127.0.0.1:3456
  - name: council-code
    provider: local
    api_model: Devstral-Small-2-24B-Instruct-2512-4bit
    api_base: http://127.0.0.1:3301
  - name: council-secure
    provider: gemini
    api_model: gemini-3.1-pro-preview
    quality_tier: 0

fallbacks:
  council-brain: [glm-51, qwen36-plus]
  council-code:  [council-27b, council-27b, qwen35-plus]

budget:
  daily_token_limit: 500000
  claude_max_daily_free: 500000
  alert_threshold_pct: 80

The smart_route flag is a scar from v0.1. The original code kept a hardcoded list of routable tiers buried in route.rs; config said one thing, code did another, and the config lost quietly for weeks while everyone wondered why tier 2 models weren’t routing. Now smart_route: bool lives in the YAML where it belongs — therapy for software.

The practical split matters more. council-brain is the default brain — Claude Opus on the Max subscription — served not from a metered Anthropic key but natively: the proxy intercepts the extracted Claude Max OAuth session token and hits the Developer API via Authorization: Bearer, so the subscription pays without a CLI wrapper. From there the router triages:

Task	Seat	Routes to
Default reasoning	council-brain	Claude Opus 4.8 (Max, via the :3456 bridge)
Code	council-code	local Devstral-Small-2-24B-Instruct-2512-4bit on :3301 (the earlier Coder-14B was retired 2026-06-07)
General chat	council-mlx	local Qwen 3.6 35B-A3B — nothing small-talk-shaped leaves the haus
Vision + security	council-secure	Gemini 3.1 Pro, pinned quality_tier: 0 for Cilghal and Mundi’s work — never rerouted

The current council-brain cross-provider fallback chain is GLM 5.2, then local offline models. Every decision is visible in config and in the JSONL usage log (~/.sanctum/metrics/token-usage.jsonl), not buried in agent prompt folklore.

Claude Code Startup Preflight

Claude Code is not trusted to have valid auth or fail quietly; that failure mode was too annoying, so it gets a dedicated startup preflight. The moving parts:

Piece	Role
ANTHROPIC_BASE_URL at https://127.0.0.1:4040	Claude Code’s entry point; proxyd speaks HTTPS with mTLS, so the session trusts the Sanctum CA via NODE_EXTRA_CA_CERTS=~/.sanctum/certs/ca.crt
com.sanctum.proxyd	the proxy runs as this system LaunchDaemon, launched through ~/.sanctum/bin/proxyd-launch, which injects the Gemini, OpenRouter, and Claude Max OAuth Session tokens from ~/.sanctum/secrets/
~/.local/bin/claude-wrapper	wraps the local claude command
tools/claude_session_preflight.sh	the wrapper runs it before executing the real Claude binary

The preflight checks whether the Claude Team token in ~/.openclaw/agents/main/agent/auth-profiles.json and its Keychain copy under anthropic-api-key are present, synchronized, and valid against Anthropic. If the token is invalid it calls tools/refresh_claude_team_token.sh --refresh:

Step	Action
setup-token	runs claude setup-token
browser	opens the Claude Team OAuth page in the operator’s default browser
code	prompts for the returned code in the same terminal session — or uses agent-browser for the hermetic automation path
sync	writes the refreshed token back to Keychain
restart	kickstarts com.sanctum.proxyd so the new token takes effect

None of this lives in the proxy binary, on purpose: the proxy should route requests, not grow a browser and roleplay as an OAuth session manager. The repair happens at the session boundary, before the first real request hits the proxy and dies with invalid x-api-key.

Note

The wrapper resolves the latest Claude binary dynamically from ~/.local/share/claude/versions/, so Claude updates do not require editing the wrapper unless the install layout itself changes.

The repair path has its own local E2E harness:

bash ~/Documents/Claude_Code/tests/test-claude-team-refresh-e2e.sh

It swaps Anthropic out for a local file:// auth page and a fake setup-token binary, then proves the URL capture, browser click-through, code handoff, token sync, and proxy restart all work end to end.

Test Coverage

The suite sits beside live checks for sanctum-mlx (:1337) and Devstral (com.sanctum.mlx-devstral-rust, :3301) native-MLX seat reachability, fallback resilience, Claude Code OAuth passthrough, and the local Claude Max auth-recovery harness. Local tiers are part of the promise, so they belong in the test count — and in the blame when they break. The tests live in the service-doctor skill, because the proxy is a service and the doctor makes haus calls — the only one in the system that does not charge by the token.

# Run the full suite
bash /Users/neo/Projects/openclaw-skills/service-doctor/tests/test-proxy.sh

# A healthy run lands around 48 pass / a couple skip; the handful of
# failures you'll see today are stale fixtures still naming retired
# models (Coder-14B, the old Gemma seat), not the proxy misbehaving.

The council session behind the streaming fix and the incident that inspired The Living Force principles are on the record elsewhere. Day to day the proxy asks for none of that drama: the door opens, the request goes where it should, the meter runs true, and when the budget runs dry the whole haus steps quietly back three feet to the Apple Silicon in the corner. Tommy worked that same door for fifteen years and never once let the wrong thing through. The binary is just keeping the shift.