LiteLLM
Open-source AI gateway: one OpenAI-format API for 100+ LLM providers
Overview
LiteLLM (by Y Combinator-backed BerriAI) is an open-source Python SDK and proxy/gateway that lets any code speaking the OpenAI format call its claimed 100+ LLM providers — OpenAI, Anthropic, Bedrock, Azure, Gemini and more — with virtual keys, budgets, fallbacks and spend tracking layered on top.
Pricing
Pricing shown for reference only. These figures reflect RECATOOLS research as of 21 Jul 2026 and may be out of date or incomplete. This is not financial or purchasing advice — always confirm the current price on the provider’s official website before making any decision.
- 100+ provider integrations, OpenAI format
- Virtual keys, budgets, rate limits
- Fallbacks, retries, load balancing
- Logging and observability callbacks
- SSO and SCIM authentication
- OIDC/JWT authorization
- Audit logs, Prometheus metrics and alerting
- 24/7 support with SLAs
What you can produce with LiteLLM
- Stand up an OpenAI-compatible proxy in front of Anthropic, Bedrock, Azure and Gemini so existing OpenAI-SDK code switches providers by changing only the model name.
- Issue virtual API keys to each team with monthly budgets and per-key rate limits, then track spend per key and per model from the admin dashboard.
- Configure automatic fallbacks so a request retries on a second provider when the primary returns a rate-limit or outage error.
- Load-balance one model alias across several Azure or Bedrock deployments using latency-based or usage-based routing strategies.
- Call any supported provider from Python with litellm.completion() and get the same response shape back, including streaming and cost tracking.
- Ship request logs, latency and cost data to Langfuse, Prometheus or OpenTelemetry through the proxy's callback hooks.
- Apply guardrails such as PII masking or content filters at the gateway so prompts are screened before they reach any provider.
ASEAN Perspective
LiteLLM in Southeast Asia
ASEAN-region availability and pricing notes coming soon. Drop the editorial team a note via /contact/ if you can supply local context (Singapore/Malaysia/Indonesia/Thailand/Vietnam).
What this is for: putting one OpenAI-format API in front of every model provider you use. LiteLLM ships as a Python SDK (litellm.completion) and a proxy/gateway server that adds virtual API keys, per-team budgets, rate limits, provider fallbacks, load-balanced routing and observability callbacks across its claimed 100+ providers — OpenAI, Anthropic, Bedrock, Azure, Gemini, Groq, local Ollama and more.
Who this is for: platform and ML-infra teams who want central control of model spend and access without maintaining their own wrapper, and individual developers who want provider portability in a single dependency. Teams that only ever call one provider, or that cannot staff the patching a self-hosted gateway needs — 2026's exploited pre-auth SQLi and the March PyPI supply-chain incident are the cautionary tales — may prefer a managed gateway instead.
Availability: free and open source to self-host. The repository is MIT-licensed except the enterprise/ directory, which carries a proprietary BerriAI license requiring a paid subscription for production use. LiteLLM Enterprise (SSO/SCIM, OIDC/JWT auth, audit logs, 24/7 SLA support, air-gapped deployment) is contact-sales pricing with a 30-day evaluation key. Installed from PyPI as litellm; Docker images cover the proxy; BerriAI is Y Combinator-backed.
What people say
Ask platform engineers what they run in front of their model providers and LiteLLM comes up before anything else. The praise is consistent across GitHub threads and community writeups: point any code that already speaks the OpenAI format at the proxy, change a model string, and the same request hits Anthropic, Bedrock, Azure or a local Ollama box. The project's own count is 100+ providers, and in practice that breadth is the moat — teams describe starting to write their own wrapper, then giving up and adopting LiteLLM for the virtual keys, per-team budgets and spend tracking alone.
Complaints cluster in three places. The proxy's YAML config has grown into dozens of interacting settings, and a serious deployment quickly acquires Postgres, Redis and Prometheus alongside it. Releases land roughly weekly on the 1.x line, so pinning versions and reading changelogs before upgrading is standard production advice. Performance gets argued about in the open: one GitHub issue documents throughput dropping from about 16 to 9 requests per second with the proxy in the path, while the maintainers' public performance roadmap and sidecar work brought measured median overhead down to single-digit milliseconds by late-1.7x builds.
2026 tested the project harder than any feature gap. In March, two PyPI releases were compromised through a stolen maintainer credential; in April, a pre-auth SQL injection (CVE-2026-42208) was exploited in the wild within roughly 36 hours of disclosure; June brought a privilege-escalation chain against the admin layer. The team published detailed timelines and patched fast, and sentiment credits them for that — but self-hosters now describe LiteLLM as infrastructure you actively operate and patch, not a container you deploy and forget.
The net read holds steady: still the fastest way to get one API, one bill and one dashboard across every provider, as long as someone on the team owns running it.
Summary of public user & expert reviews, compiled by RECATOOLS.
About this listing
This entry was compiled from publicly available data including LiteLLM's official website, press releases, documentation, and reputable third-party publications. RECATOOLS is not affiliated with LiteLLM unless explicitly stated.
Third-party AI tools update their pricing, features, availability, and policies frequently. Information here may be outdated by the time you read this — we make reasonable efforts to keep listings current, but cannot guarantee absolute accuracy.
For the latest details, please refer to LiteLLM directly →
Spotted something out of date? Suggest an update →
More in Code & Dev Tools
