Published by Winding AI, Inc., the company that makes Griffin. No vendor paid to be listed.Fees read November 19, 2026 · About this site
bestllmgateway.comSee the gateways
13 gateways read on November 19, 2026

The best LLM gateway, by the fees it adds and the jobs it does

If you run your own servers, the best LLM gateway is LiteLLM or Bifrost: both are free, open source and do all six jobs below. If you want an AI gateway hosted for you, Cloudflare AI Gateway does five of the six and charges nothing on your own provider keys.

OpenRouter has the most models (600+) on one bill, for 5.5% on credits. Vercel AI Gateway takes no markup at all. Griffin isn't a gateway: it runs on one to cut what each feature spends.

  • Vendor pages read on one day
  • Every fee worked out on the same $10,000 month
  • No vendor paid to be listed

Updated November 20, 2026.

Which one

Pick by the job you need done

One line per need; the cards say where each falls short.

You run your own servers and want every job free

→Bifrost or LiteLLM

You want it hosted, cached, with no fee on your keys

→Cloudflare AI Gateway

You want 600+ models on one bill

→OpenRouter

You want open source now and a hosted plan later

→LLM Gateway

You run Kubernetes and Envoy already

→Agent Router

You need budget rules for many teams

→TrueFoundry

You need data kept in the EU by default

→Requesty

You use OpenRouter, Vercel or Cloudflare and want each feature's cost cut

→Griffin
The basics

What an LLM gateway does

One API in front of every model provider. It holds the keys, retries on another provider when one fails, serves repeats from cache, stops a key at its budget and logs the call. Also called an AI gateway or an LLM proxy.

App to gateway to four providers Your app LLM gateway keysfallbackscachebudgetsrate limitslogs Provider A Provider B Provider C Provider DApp to gateway to four providers Your app LLM gateway keysfallbackscachebudgetsrate limitslogs Provider A Provider B Provider C Provider D
Six jobs are compared on this site: one API for many providers, fallbacks, caching, budgets, rate limits, and a build you can self-host.
Compare

LLM gateways side by side

From vendor pages on November 19, 2026. “Exact” caching repeats identical answers; “semantic” similar ones.

Ten options, in this page's order Own keys = bring your own provider keys
GatewayFee on bought creditsYour own keysFallbacksCachingBudgetsRate limits you setSelf-host
BifrostNo hosted planBuilt inYesExact + semanticYesYesYes, Apache-2.0
LiteLLMNo hosted planBuilt inYesExact + semanticYesYesYes, MIT core
Cloudflare AI Gateway5% (Unified Billing)No feeYesExactYesYesNo
LLM Gateway5% (+1.5% intl. cards)No feeYesExact, 60 sYesNot statedYes, AGPLv3
TrueFoundryNo token resalePlan fees onlyYesExact + semanticYesPro and upEnterprise only
Agent RouterNo hosted planBuilt inYesNoToken quotasYesYes, Apache-2.0
OpenRouter5.5% (8% Business)Free to $25k a month, then 5%YesExact, up to 1 dayYesNoNo
Vercel AI Gateway0%Free with fallback onYesNoYesNoNo
Requesty5% on model cost0% (per llms.txt)YesNoYesNoNo
GriffinDoesn't resell tokensAlways your keysNoNoNoNoNo (MIT SDK)

Swipe the table sideways to see every column.

The gateways

Ten options, one card each

Order: by how many of the six jobs each does without an enterprise contract, then by the lower fee on a $10,000 month (with your own keys, then when buying its credits), then alphabetically. Griffin does none of the six, so it is last.

1 · 6 of 6 jobs

Bifrost

Best for a fast self-hosted gateway

$0open source; you run it

Maxim's open-source gateway in Go: one OpenAI-style API, fallbacks, exact and semantic caching, and budgets and rate limits on virtual keys, all free. Enterprise (custom price) adds clustering.

Good

  • Claims 11 µs of overhead per request at 5,000 a second (on mocked calls)
  • SQLite by default: no database to run on day one

Watch out

  • No hosted plan: you run and scale it
  • Caching needs a vector store such as Redis or Qdrant
2 · 6 of 6 jobs

LiteLLM

Best for self-hosting with the most providers

$0MIT core; Enterprise priced to your scale

The most-starred open-source gateway (63,588 GitHub stars): one OpenAI-format API to 100+ providers, with virtual keys, budgets, rate limits, fallbacks and caching free. Enterprise is annual, “never per token”.

Good

  • Spend by key, user, team and org without a licence
  • Publishes 8 ms p95 overhead at 1,170 requests a second (four instances)

Watch out

  • Virtual keys need Postgres; Redis is advised at scale
  • Code in litellm/enterprise/ needs a paid licence in production
3 · 5 of 6 jobs

Cloudflare AI Gateway

Best hosted, with no fee on your own keys

$0core features; 5% only on bought credits

A hosted gateway on Cloudflare's network. Analytics, caching, rate limits, fallbacks and spend limits are free on every plan; only credits bought through Unified Billing carry a 5% fee.

Good

  • No fee on your own provider keys
  • Spend limits by provider, model, user or team

Watch out

  • No self-hosting; exact-match caching only so far
  • Log allowances follow your Workers plan and change on 1 December 2026
4 · 5 of 6 jobs

LLM Gateway

Best for open source that is also hosted

$0own keys; 5% when you buy credits

An AGPLv3 gateway you can self-host or use hosted. Hosted, it takes 5% when you buy credits (plus 1.5% on international cards) and nothing on your own keys; 200+ models from 40+ providers.

Good

  • Free exact-match response cache, 60 seconds by default
  • Lifetime and monthly spend limits per key

Watch out

  • The free plan logs metadata only; full payloads are Enterprise
  • Self-hosting needs Postgres and Redis; ee/ features are commercial
5 · 5 of 6 jobs

TrueFoundry AI Gateway

Best for budgets across many teams

$25a user a month on Pro, then $20 per 100k requests

A hosted gateway with budget rules, rate limits, caching and latency routing. Developer is free for 3 users at 10,000 requests each a month; Pro is $25 a user with 20,000 each. No token markup.

Good

  • Budgets scoped to users, teams, models or metadata
  • Its docs measure +7 ms on one 1-CPU pod

Watch out

  • Self-hosting (VPC, on-prem, air-gapped) is Enterprise only
  • Request fees add up: about $425 at 2 million requests on one seat
6 · 4 of 6 jobs

Agent Router (formerly Envoy AI Gateway)

Best for Kubernetes and Envoy shops

$0open source; hosted versions from other firms

Envoy AI Gateway took this name when it moved to the Agentic AI Foundation in September 2026; same Apache-2.0 code. Runs on a laptop or Kubernetes, with 22 providers, fallbacks and token rate limits.

Good

  • Provider keys and cloud credentials stay in your cluster
  • Token quotas per model, with cost weights

Watch out

  • No response caching; quotas are tokens, not dollars
  • Rate limits need a Redis instance
7 · 4 of 6 jobs

OpenRouter

Best for the most models on one bill

5.5%on credits you buy; own keys free to $25k a month

A hosted API to 600+ models from 80+ providers. No markup on inference; 5.5% when you buy credits (8% on Business). Own keys: the first $25,000 of use a month is free, then 5%.

Good

  • The most models here, with automatic provider fallback
  • Credit caps per key and workspace budgets

Watch out

  • No self-hosting and no rate limits you set yourself
  • Response caching is exact-match, for a day at most
8 · 3 of 6 jobs

Vercel AI Gateway

Best for zero markup on credits

0%markup on tokens; add-ons priced apart

Vercel's hosted gateway to 400+ models, usable from any cloud. No markup on tokens; budgets per team, project, key or member. Own keys are free while fallback to Vercel's keys is on.

Good

  • The only hosted option here with no fee on bought credits
  • Managed fallbacks, load balancing and retries

Watch out

  • No response caching: Vercel says to cache in your app
  • The free monthly credit has no published dollar amount
9 · 3 of 6 jobs

Requesty

Best for EU data residency by default

5%on model cost; 0% on your own keys

A managed gateway to 600+ models from 30+ providers, with data kept in the EU by default. It adds 5% to model cost and, per its llms.txt, nothing on your own keys. Free: 200 requests a day.

Good

  • Spend limits per key, fallbacks and latency routing on every plan
  • EU data residency on every plan

Watch out

  • No response caching (provider prompt caching only)
  • No self-hosting; no per-key rate limits to set
10 · 0 of 6 jobs

Griffin

Best for cutting cost on OpenRouter, Vercel or Cloudflare

$0to watch; Route is 5% of routed spend

Not a gateway: a TypeScript fetch wrapper from Winding AI that sits on what you use (OpenRouter, Router.com, Vercel or Cloudflare AI Gateway, or nine providers direct). It records each call's cost by use case, replays real calls on cheaper models with a judge (replay grading), and routes a use case that passed on your own OpenRouter key.

Price: Watch $0. Test: $10 of replays free, then $20 packs. Route: 5% of routed spend after the first $50.

Good

  • Griffin routes each feature to a cheaper model only after that model passes replays of the app's own calls
  • Your key, your provider bill; calls go straight to the provider if Griffin is down

Watch out

  • Does none of the six gateway jobs: keep a gateway for those
  • Routed calls take one extra hop through Griffin's Cloudflare edge

On the data page, not in the ten: Kong AI Gateway (AI fallbacks, cache and token limits need its Enterprise licence), Portkey (pricing page now a Palo Alto Networks sales form) and Helicone (maintenance mode since March 2026).

Fees

What each adds to a $10,000 month

The sample: $10,000 of model spend in a month, 2 million requests, one seat where seats are billed, list prices.

GatewayWith your own keysBuying its credits
Bifrost$0 + your serversNo hosted plan
LiteLLM$0 + your serversNo hosted plan
Cloudflare AI Gateway$0$500 (5%)
LLM Gateway$0$500 (5%), $650 on an international card
TrueFoundryAbout $425 (1 seat)No token resale
Agent Router$0 + your serversNo hosted plan
OpenRouter$0 (under $25,000 a month)$550 (5.5%)
Vercel AI Gateway$0$0 (no markup)
Requesty$0$500 (5%)
Griffin$0 to watch; Route 5% of what it routesDoesn't resell tokens

Self-hosted gateways cost what their servers cost you. Kong, Portkey, Helicone and the arithmetic are on the data page.

Hosted or not

How to choose between hosted and self-hosted

  • Self-host if keys must stay insideLiteLLM, Bifrost, Agent Router and LLM Gateway keep keys in your network. You pay in servers and databases; Bifrost starts on SQLite.
  • Go hosted if nobody wants a pagerCloudflare, OpenRouter, Vercel, Requesty and TrueFoundry run it for you. On four of them your own keys cost nothing.
  • Read overhead claims as marketingBifrost claims 11 µs, LiteLLM 8 ms p95, TrueFoundry 7 ms, Portkey 20 to 40 ms, each on its own test. Cloudflare, OpenRouter and Vercel publish none.
  • Keep the API standardEvery gateway here speaks an OpenAI-style API, so moving is mostly a base URL and a key.
FAQ

Questions about AI gateways

What is an LLM gateway?

An LLM gateway (also called an AI gateway or LLM proxy) is one API between your app and the model providers. It holds the provider keys and adds fallbacks when a provider fails, caching, spend budgets, rate limits and logs to every call.

What is the best open-source LLM gateway?

LiteLLM and Bifrost do all six jobs here in their free builds. LiteLLM has 63,588 GitHub stars and 100+ providers; Bifrost is Go, Apache-2.0.

How much does Cloudflare AI Gateway cost?

Its core features (analytics, caching, rate limiting) are free on every plan. Credits bought through Unified Billing carry a 5% fee; provider prices pass through with no markup.

Is LiteLLM free?

The gateway is MIT-licensed and free to self-host, virtual keys, budgets and rate limits included. Code in litellm/enterprise/ needs a paid BerriAI licence in production.

Is OpenRouter an LLM gateway?

As a hosted service, yes: one API to 600+ models, provider fallback, response caching and per-key credit caps. It can't be self-hosted and charges 5.5% on credits you buy.

What happened to Portkey?

Palo Alto Networks closed its purchase of Portkey on 29 May 2026 and sells it as Prisma AIRS AI Gateway; portkey.ai/pricing now redirects to a sales form. It is on the data page.

Does Griffin replace a gateway?

No. Griffin runs on top of OpenRouter, Router.com, Vercel or Cloudflare AI Gateway, or direct provider keys, and moves use cases that passed replay grading to cheaper models. Winding AI, Inc., which publishes this site, makes it.

Who publishes this, and how it was checked

Published by Winding AI, Inc., the company that makes Griffin. Winding AI earns money when teams pay for Griffin, and from nothing else on this site: no vendor paid to appear, and no link pays a commission. Griffin has the same card, row and order rule as every gateway, and is last because it does none of the six gateway jobs.

How this page was checked: on November 19, 2026 we read each gateway's pricing page, plan table, feature docs and licence file, and took stars and releases from the GitHub API. The $10,000 month is our arithmetic on list prices. Sources are on the data page; corrections to support@winding.ai.