Open source AI gateway licences, read closely
Permissive, with a paid folder. LiteLLM's core is MIT, but code under litellm/enterprise/ may only run in production with a BerriAI Enterprise licence. Bifrost is Apache-2.0 and ships separate enterprise builds. LLM Gateway is AGPLv3, which obliges you to publish changes if you offer it as a service, with its ee/ folder under a commercial licence.
Permissive, with paid plugins. Kong Gateway is Apache-2.0 and its open-source repo carries six AI plugins, including the basic AI proxy. The advanced proxy (load balancing, fallbacks), the semantic cache and token-based rate limiting are labelled “AI License Required”.
Plain permissive. Agent Router (formerly Envoy AI Gateway) is Apache-2.0 throughout; Portkey's gateway is MIT, though its last release was in January 2026 and its semantic cache stays in the hosted product.
A quiet change. Helicone's Rust gateway switched its licence file from Apache 2.0 to GPL v3 in November 2025 while its README still says Apache, and it has had no stable release.
How to choose one
- All six jobs with no licence: LiteLLM or Bifrost. Both give you virtual keys with budgets and rate limits, fallbacks and caching in the free build.
- Already on Kubernetes and Envoy: Agent Router plugs into Envoy Gateway and keeps cloud credentials in the cluster.
- Want a hosted plan as a fallback: LLM Gateway runs the same code at llmgateway.io (5% on bought credits, nothing on your own keys).
- Already run Kong for APIs: the free AI proxy is there today; budget for the Enterprise licence if you need fallbacks or token limits.
- Count the databases. Bifrost starts on SQLite; LiteLLM needs Postgres for virtual keys; LLM Gateway needs Postgres and Redis; Agent Router needs Redis once you rate-limit.
LLM proxy or LLM gateway?
The words are used for the same thing: a server that speaks an OpenAI-style API to your app and calls the providers behind it. “Proxy” tends to mean the thin version (one API and key handling, as in LiteLLM's original proxy); “gateway” the version with budgets, limits, caching and logs. Every project above does at least the proxy part.
If you'd rather not run one
Hosted gateways compared on the home page (Cloudflare AI Gateway, OpenRouter, Vercel AI Gateway, Requesty, TrueFoundry) do the same jobs for a fee or a cut of credits; the fees data page prices all 13 on one $10,000 month. Griffin is a different kind of tool: not a gateway but a TypeScript wrapper that records cost by use case and tests cheaper models. Its supported list is OpenRouter, Router.com, Vercel AI Gateway, Cloudflare AI Gateway and direct provider keys, so it doesn't list the self-hosted gateways above. Its SDK is MIT on npm; the service is hosted. Griffin is made by Winding AI, Inc., which publishes this site.
Questions
What is the most popular open source LLM gateway?
LiteLLM, with 63,588 GitHub stars on 19 November 2026. Kong Gateway has 44,400, but that counts the whole API gateway, not only its AI plugins; Portkey's gateway has 13,420 and Bifrost 9,794.
Is an open source AI gateway really free?
The code is. Running it costs servers and often Postgres or Redis, and four of the seven keep some features in a paid folder or licence: LiteLLM, Kong, Bifrost and LLM Gateway.
Can I use an LLM proxy with my own provider keys?
Yes: a self-hosted gateway only ever uses your keys, and they stay on your network.