teerouter
Request access

Private alpha · in development

Confidential AI that keeps running.

TeeRouter is a gateway you run next to your app. It checks each provider’s hardware attestation, encrypts every request so only that provider’s enclave can read it, and fails over to another verified provider when one goes down. Outside your own systems, only the verified enclave can read your prompts. Our servers never see them.

Request early access See how it works
  • OpenAI and Anthropic APIs
  • Your own provider keys
  • Runs on macOS and Linux

Simulated request flow

A simulation of two requests from your app through the TeeRouter gateway. The first is verified, sealed to Tinfoil’s enclave and answered. For the second, Tinfoil’s route returns 503. The gateway skips a standard API route that has no enclave, seals the request to Privatemode’s pre-verified enclave and gets the answer. TeeRouter’s cloud only supplies the signed catalog and route health.

Simulation. Route names and model builds are real. Timings, prompts and log lines are illustrative.

Where your prompt is readable

Outside your systems, only the enclave reads it.

Every AI request is readable somewhere. What matters is where, and who could misuse it. Each icon answers one question for one party: how exposed are you if it turns malicious?

Lab API or standard router

TLS protects each hop, then each service reads the prompt.

  1. Your sideYour app
  2. Router
  3. Provider API
  4. Model server

Outside your systemsReadable by the router and provider operators

Attested gateway

A hosted gateway decrypts every request inside its TEE, then forwards it. Its operator controls that code.

  1. Your sideYour app
  2. Gateway TEEthird-party code
  3. Provider TEE

Outside your systemsReadable in the gateway’s TEE, for every request, and in the provider’s TEE

If the gateway is compromised, or its operator ships a bad release, every prompt it handles is exposed. Its checks of the provider fall with it.

TeeRouter

The gateway runs on your side, verifies the provider and seals the request. The provider’s enclave is the only place it opens.

  1. Your sideYour appTeeRouter gateway
  2. Provider TEE

Outside your systemsReadable only in the provider’s TEE

TeeRouter’s cloud supplies the signed catalog and route health. It never carries a request.

  • Sees plaintext. Can read, keep or alter everything.
  • TEE-bounded. To read content, it has to break the TEE or get bad code approved.
  • Metadata only. Can delay or observe traffic.
  • Your side. Your app, your hosts and the gateway you run.

The market map compares 20 routers, gateways and providers on these boundaries, with sources.

Verification-aware failover

A provider can be up and still unsafe to use.

Confidential routes fail like any API: timeouts, errors, rate limits. They also fail verification, when evidence expires, a new release isn’t endorsed yet or firmware falls below your floor. A generic AI gateway only sees the first kind. TeeRouter sees both and routes around them.

1 of 91

confidential inference routes passed every check in a public attestation monitor snapshot.

  • Fully verified1
  • Partially verified50
  • Failed25
  • Unreachable15

Source: confidentialinference.net attestation monitor, run of 10 October 2026.

How a generic AI gateway and TeeRouter handle route problems
When this happensGeneric AI gatewayTeeRouter
A provider times out, errors or rate-limitsRetries another providerDemotes the route and fails over
Evidence expires or firmware drops below your floorDoesn’t noticeRoute is ineligible until it verifies again
The provider ships a release you haven’t approvedKeeps sendingWaits for your approval or an endorser you named
The backup serves a different build of the modelDoesn’t checkFails over only within a model class you approved
The backup decrypts outside an enclaveDoesn’t checkRefuses it

Standby routes are verified in advance, so a failover doesn’t wait for a fresh attestation. TeeRouter publishes parity results for each pair of builds it lets you approve as one class.

How a request travels

Verify, seal, send.

All checks run in the gateway on your side, before your prompt leaves your network.

  1. 1

    Verify

    The gateway checks each route’s attestation against your policy. It confirms the hardware is genuine, the firmware is current and the workload is a release you trust. Evidence is cached and refreshed before it expires.

    • vendor roots
    • TCB level
    • freshness
    • workload
    • GPU evidence
    • weights digest
  2. 2

    Seal

    It encrypts the request to a key that is bound into that attestation. Only the enclave that produced the evidence can open it. The response comes back encrypted and authenticated the same way.

  3. 3

    Send

    It calls the provider directly, with your own provider key. No TeeRouter server sits on the path, so a breach of our servers can’t expose your content.

  4. 4

    Fail over

    If a route fails before the first byte arrives, the gateway retries on a standby route that passed the same checks. Errors after streaming has started are reported, not retried, and nothing is retried after a tool call.

For developers

Change one URL.

TeeRouter serves the OpenAI Chat Completions API and the Anthropic Messages API on localhost. Point an SDK, a coding agent or your AI gateway at it.

  • Provider keys go in with teerouter keys add, never into your app or agent config.
  • Every response names the route that served it and what was proven about it.
  • teerouter why explains a routing decision, including every route it skipped and why.
  • Logs, metrics and the evidence ledger carry no prompt or response content.

Works withOpenAI SDKsAnthropic SDKsLiteLLMPortkeyKongCoding agents

Terminal
$ teerouter init
$ teerouter keys add tinfoil
$ teerouter keys add privatemode
$ teerouter up
# OpenAI-compatible API on http://127.0.0.1:8080/v1
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="local")

raw = client.chat.completions.with_raw_response.create(
    model="glm-5.3",
    messages=[{"role": "user", "content": "Review this clause."}],
)
print(raw.headers["x-teerouter-route"])     # tinfoil/glm-5.3-inferact
print(raw.headers["x-teerouter-failover"])  # false
reply = raw.parse()
Response headers · example
HTTP/1.1 200 OK
X-TeeRouter-Request-ID: 5f1c0e9a7b2d48e3a6c4f0b19d82e7a1
X-TeeRouter-Route: tinfoil/glm-5.3-inferact
X-TeeRouter-Assurance: [3,3,3,3,3,2,2,3,1,1,1]
X-TeeRouter-Evidence: 7d2e…9b41
X-TeeRouter-Failover: false

When nothing verifies, you get a refusal.

TeeRouter never falls back to a weaker route to keep a request alive. The error lists each route and the check it failed, so you can fix the cause.

HTTP/1.1 503 Service Unavailable

{"error": {"message": "tee_policy_unsatisfied", "type": "server_error",
  "param": null, "code": "tee_policy_unsatisfied",
  "route_reasons": [
    {"route": "tinfoil-glm", "reason": "evidence_expired"},
    {"route": "privatemode-glm", "reason": "unavailable"}]}}

For your security review

Evidence your reviewer can sign off on.

Start from a preset or write your own policy. Every routing decision lands in a hash-chained evidence ledger that you can export for auditors with teerouter evidence export.

standard

For trials and low-sensitivity data

Verification
Provider SDK level, current release
Plaintext
Hosted mode allowed
Redundancy
Best effort
Region
Any
strictRecommended

For confidential business data and proprietary code

Verification
To hardware vendor roots, GPU evidence included
Releases
Endorsed by TeeRouter review and the provider
Model
Weights digest pinned
Plaintext
Only inside the inference enclave
Redundancy
2 or more independent operators per model class
strict-eu

For GDPR-sensitive workloads

Verification
Same as strict
Region
EU infrastructure, as asserted by the operator
Redundancy
One qualifying operator today. Responses and policy checks say so.

Nothing we publish can widen what your gateway trusts.

TeeRouter publishes a signed catalog, release endorsements and model parity results. Your gateway uses them only to narrow which routes are eligible. Approving a new release takes a signature that your own policy names.

  • pinnedYou approve every new release yourself.
  • reviewedYour policy names who may approve releases, for example 2 of: provider, TeeRouter review.
  • provider-latestFollows the provider alone. Not allowed in strict.

What attestation doesn’t prove

We state the limits up front, so your review doesn’t have to find them.

  • That the attested code is bug-free or that its output is safe.
  • That there are no side channels, such as cache-timing leaks between users.
  • Where the hardware physically is. Location is the operator’s claim unless a cloud endorsement covers it.
  • GPU state beyond what the attested workload reports.

Routes

One model, independent operators.

Failover needs a second operator serving the same model. For a handful of open-weight models that already exists. TeeRouter starts with Tinfoil and Privatemode and adds operators once their verification checks out.

Open-weight models served by two or more independent confidential operators
ModelOperatorsNotes
GLM-5.3Tinfoil · PrivatemodeDefault model. Two different NVFP4 builds, so you approve both before failing over between them.
gpt-oss-120bTinfoil · PrivatemodeIdentical weights at both operators.
Kimi K3Tinfoil · ChutesNEAR AI also lists it, without operator attribution.
GLM-5.3-FlashPrivatemode · PhalaPhala’s dstack serves it for NEAR AI and Venice.
GLM-5.2Chutes · Privatemode—

Operators per model from the confidentialinference.net monitor, 10 October 2026. Of the 58 models it listed, 53 had a single attributed operator.

Early access

Join the private alpha.

We’re working with a small number of teams that need confidential inference in production. You bring your own provider keys. We help you set up routes, policy and failover, and we want to hear where it breaks.

Request early access hello@teerouter.com
  1. In development

    Private alpha

    Local gateway with OpenAI and Anthropic APIs, Tinfoil and Privatemode routes, verification-aware failover and a signed catalog.

  2. Next

    Public launch

    Self-serve sign-up, a hosted endpoint, a Helm chart and sidecar mode, more operators and an optional ciphertext-only relay.

  3. Later

    Enterprise

    Invoicing, SSO and SCIM, signed policy as code, and frontier confidential endpoints as routes.

Questions

Common questions

Something missing? Ask us at hello@teerouter.com.

Can TeeRouter read my prompts?

Our servers can’t. In direct mode, the default, the gateway runs on your infrastructure and sends each request straight to the provider’s enclave, sealed to that enclave’s key. TeeRouter’s servers handle your account, the signed catalog and, if you opt in, route health metadata. They never receive prompts or responses.

The gateway itself is our code running on your side, like any software you install. A malicious release could read your prompts, so the gateway never updates itself. You decide which release runs.

The planned hosted endpoint is different. It decrypts every request inside TeeRouter’s attested gateway, and if that gateway were compromised, every prompt it handles would be exposed. Hosted responses are labeled, and strict policies refuse hosted mode.

What happens when every verified route is down?

The request fails with HTTP 503 and a tee_policy_unsatisfied error that lists each route and why it was skipped. TeeRouter never sends your prompt to an unverified or non-confidential route to keep a request alive.

Can I fall back to a self-hosted or external model?

Yes, but outside TeeRouter. A fallback to non-attested inference, such as your own servers or an ordinary hosted API, is a legitimate choice. It isn’t something TeeRouter does, because TeeRouter only routes to verified confidential routes.

We recommend defining fallbacks in a dedicated load balancer or AI gateway in front of TeeRouter, such as LiteLLM, Portkey or Kong. When TeeRouter returns tee_policy_unsatisfied, that gateway decides whether the workload may move to a non-attested route. The decision stays explicit and auditable on your side.

Do I have to trust TeeRouter’s endorsements?

No. Remote inputs, including our catalog, endorsements and health feed, can only narrow what your gateway accepts. With the pinned channel you approve every release yourself. With reviewed, your policy names which signers may approve new releases, for example two of: the provider and TeeRouter review.

What does attestation actually prove?

That genuine confidential computing hardware is running a specific, measured software release, and that the encryption key belongs to that environment. It doesn’t prove the code is bug-free, that there are no side channels, or where the server physically sits. Each response reports these properties separately in X-TeeRouter-Assurance.

Where does inference run?

On each provider’s confidential computing hardware: CPU enclaves such as AMD SEV-SNP or Intel TDX, with NVIDIA GPUs in confidential mode. strict-eu limits routes to EU infrastructure as asserted by the operator. Today one operator qualifies, and TeeRouter tells you when a policy has no second operator to fail over to.

What does it cost?

In the alpha you bring your own provider keys and pay providers directly. TeeRouter will charge a subscription for route health, release endorsements and parity testing. We’re setting prices together with our design partners.

Is the gateway open source?

The gateway is licensed under Apache-2.0. The repository is private while it’s in development, and we plan to publish it as open source.