Published by Winding AI, Inc., the company that makes Griffin. No vendor paid to be listed.Fees read November 19, 2026 · About this site
bestllmgateway.comSee the gateways

Best LLM gateway / Open source

Read on November 19, 2026

Open source LLM gateways: seven you can run yourself

An open source LLM gateway (or LLM proxy) runs on your own servers, so provider keys and prompts never leave your network and no one takes a cut of your model spend. Seven are maintained in public today. Their licences differ more than their feature lists: four keep some features in a paid folder or licence, and one changed licence quietly.

Updated November 20, 2026. Stars and releases from the GitHub API on November 19, 2026 (release dates in UTC).

The seven, by GitHub stars

ProjectLicence and paid partsStarsLatest releaseLanguageNeeds
LiteLLMMIT; litellm/enterprise/ needs a BerriAI licence63,588v1.110.2, 2026-11-20PythonPostgres for virtual keys; Redis at scale
Kong GatewayApache-2.0; AI fallbacks, semantic cache and token limits need AI Gateway Enterprise44,4003.9.5, 2026-11-19LuaDB-less mode or Postgres
Portkey GatewayMIT; semantic cache only in hosted and enterprise versions13,420v1.15.2, 2026-01-12TypeScriptNo database step (npx)
BifrostApache-2.0; clustering, adaptive load balancing and guardrails are Enterprise9,794transports/v2.6.0, 2026-11-19GoSQLite by default; vector store for caching
Agent RouterApache-2.02,423v1.3.0, 2026-11-18GoKubernetes or aigw run; Redis for limits
LLM GatewayAGPLv3; ee/ under a commercial licence1,753v1.26.0, 2026-11-16TypeScriptPostgres and Redis
Helicone AI GatewayGPL-3.0 since November 2025 (README still says Apache)647None stable; last beta 2025-07-21RustRedis optional

Swipe the table sideways to see every column.

Open source AI gateway licences, read closely

Permissive, with a paid folder. LiteLLM's core is MIT, but code under litellm/enterprise/ may only run in production with a BerriAI Enterprise licence. Bifrost is Apache-2.0 and ships separate enterprise builds. LLM Gateway is AGPLv3, which obliges you to publish changes if you offer it as a service, with its ee/ folder under a commercial licence.

Permissive, with paid plugins. Kong Gateway is Apache-2.0 and its open-source repo carries six AI plugins, including the basic AI proxy. The advanced proxy (load balancing, fallbacks), the semantic cache and token-based rate limiting are labelled “AI License Required”.

Plain permissive. Agent Router (formerly Envoy AI Gateway) is Apache-2.0 throughout; Portkey's gateway is MIT, though its last release was in January 2026 and its semantic cache stays in the hosted product.

A quiet change. Helicone's Rust gateway switched its licence file from Apache 2.0 to GPL v3 in November 2025 while its README still says Apache, and it has had no stable release.

How to choose one

  1. All six jobs with no licence: LiteLLM or Bifrost. Both give you virtual keys with budgets and rate limits, fallbacks and caching in the free build.
  2. Already on Kubernetes and Envoy: Agent Router plugs into Envoy Gateway and keeps cloud credentials in the cluster.
  3. Want a hosted plan as a fallback: LLM Gateway runs the same code at llmgateway.io (5% on bought credits, nothing on your own keys).
  4. Already run Kong for APIs: the free AI proxy is there today; budget for the Enterprise licence if you need fallbacks or token limits.
  5. Count the databases. Bifrost starts on SQLite; LiteLLM needs Postgres for virtual keys; LLM Gateway needs Postgres and Redis; Agent Router needs Redis once you rate-limit.

LLM proxy or LLM gateway?

The words are used for the same thing: a server that speaks an OpenAI-style API to your app and calls the providers behind it. “Proxy” tends to mean the thin version (one API and key handling, as in LiteLLM's original proxy); “gateway” the version with budgets, limits, caching and logs. Every project above does at least the proxy part.

If you'd rather not run one

Hosted gateways compared on the home page (Cloudflare AI Gateway, OpenRouter, Vercel AI Gateway, Requesty, TrueFoundry) do the same jobs for a fee or a cut of credits; the fees data page prices all 13 on one $10,000 month. Griffin is a different kind of tool: not a gateway but a TypeScript wrapper that records cost by use case and tests cheaper models. Its supported list is OpenRouter, Router.com, Vercel AI Gateway, Cloudflare AI Gateway and direct provider keys, so it doesn't list the self-hosted gateways above. Its SDK is MIT on npm; the service is hosted. Griffin is made by Winding AI, Inc., which publishes this site.

Questions

What is the most popular open source LLM gateway?

LiteLLM, with 63,588 GitHub stars on 19 November 2026. Kong Gateway has 44,400, but that counts the whole API gateway, not only its AI plugins; Portkey's gateway has 13,420 and Bifrost 9,794.

Is an open source AI gateway really free?

The code is. Running it costs servers and often Postgres or Redis, and four of the seven keep some features in a paid folder or licence: LiteLLM, Kong, Bifrost and LLM Gateway.

Can I use an LLM proxy with my own provider keys?

Yes: a self-hosted gateway only ever uses your keys, and they stay on your network.

Published by Winding AI, Inc., the company that makes Griffin. Licences read from each repo's LICENSE file and plugin pages on November 19, 2026; corrections to support@winding.ai.