Updated

LiteLLM

Open-source proxy and Python SDK giving a single OpenAI-compatible API to call 100+ LLM providers, with routing, retries and spend tracking.

21 makers' products we track use LiteLLM. For LLM API it's 6th most chosen of 13 options: 21 of the 941 makers' products we know the LLM API of (2%). Among the 263 launched in 2026, 3% chose it, against 1% of those launched in 2024 or earlier. 139 open-source projects also declare it in their code. What makers like most: One OpenAI-compatible API across providers lets teams switch or add models without rewriting application code. Their most common complaint: The codebase and dependency tree are heavy, and the proxy can use gigabytes of memory.

Open sourceSelf-hostableFree tierPaid from Contact sales

Profile checked

Works with AI agents:llms.txtMCP
Ask your AI about this, with this page as the source:ChatGPT ↗Claude ↗Perplexity ↗

How makers use LiteLLM

Makers' products we track use LiteLLM for LLM API (21).

What LiteLLM is used for

LLM API · Gateway (many providers, one API) →Running your own OpenAI-compatible proxy in front of your own provider keys, with retries, routing and per-key budgets.Trade-off: You deploy, monitor and upgrade it yourself; SSO and audit logs are in the paid enterprise tier.
LLM Gateway & Routing · Self-hosted proxy →Running your own OpenAI-compatible proxy in front of your own provider keys, with retries, routing and per-key budgets.Trade-off: You deploy, monitor and upgrade it yourself; SSO and audit logs are in the paid enterprise tier.
Kong Gateway2Helicone1Unify1OpenRouter0LiteLLM0Makers' products on each, for LLM gateway & routing

LiteLLM vs the alternatives makers chose

For LLM API, 21 of the 941 makers' products we know the choice of picked LiteLLM (2%), behind OpenAI API (78%). Among those launched in 2026 it's 3%.

OptionMakers' products2026 launchesFree forAt 10,000 usersOpen source
OpenAI API736 78%75%~1,000 users$10/mo · GPT-6 LunaNo
Claude252 27%21%~1,000 users$10/mo · Claude Haiku 5.5No
Gemini API99 11%11%~360 users$28/mo · Gemini 3.1 Flash-LiteNo
Groq39 4%3%~1,500 users$6.75/mo · GPT OSS 20BNo
Azure OpenAI22 2%0%——No
LiteLLM21 2%3%——Yes

For LLM API: how many of the 941 makers' products we know the LLM API of chose each option, and the share among the 263 launched in 2026, since a tool that came first has products from every year, and what each costs at 10,000 users.

Costs are the cheapest plan that fits, from each vendor's pricing page, at the usage our stack planner assumes for a product with that many users. Free for: the users its cheapest plan a business can use stays free for.

When to use LiteLLM, and when not

Pick LiteLLM if

  • You want that routing and spend control but self-hosted, with keys and logs on your servers. LLM API →
  • Provider keys and request data must stay on your own servers. LLM Gateway & Routing →

Something else fits better if

  • You want the strongest general models and the widest ecosystem of examples and integrations: OpenAI API or Claude or Gemini API.
  • Token cost dominates your budget, for example high-volume batch or agent workloads: DeepSeek or Groq or Mistral AI.
  • You want to try or mix many models with one key and one bill, and fall back when a provider is down: OpenRouter.
  • You want to try or offer many models with one key and one bill, today: OpenRouter or Vercel AI Gateway.
  • You already deploy on Vercel or use the AI SDK: Vercel AI Gateway.
From each decision's guide, written from what makers' products chose and why.

LiteLLM, from BerriAI, is an MIT-licensed project with two parts: a Python library that calls 100+ LLM providers through one function and always returns OpenAI-shaped responses, and an AI gateway (proxy) you run yourself that offers the same thing as an OpenAI-compatible HTTP API for any language.

More about LiteLLM

With the library you call litellm.completion(model="provider/model", ...), and its Router adds retries, load balancing and fallbacks between named model groups. The gateway is a Docker image: you register models and provider credentials in a config.yaml or the built-in Admin UI, then hand out virtual keys to apps and teammates. Postgres holds models, keys and spend logs, and the gateway needs a master key and a salt key that encrypts stored credentials. Clients such as the OpenAI or Anthropic SDKs, Claude Code or Codex connect by changing their base URL.

Useful for a small team: budgets and tpm/rpm limits per key, user or team; automatic failover to another provider after retries; logging to Langfuse, LangSmith, OpenTelemetry, S3 or Datadog; and no telemetry sent to LiteLLM when self-hosted. It runs with Docker Compose, one-click Railway or Render templates, Helm on EKS, GKE or AKS, or Terraform modules for AWS and GCP.

The main limit is that you operate it: a database, upgrades, and a salt key that cannot be rotated in place. Budgets are not enforced without the database, and SSO beyond five users, audit logs and multi-region deployment need an enterprise license.

How LiteLLM itself is built

28 tools, from its own code, website and Product Hunt page.

LiteLLM pros and cons

What makers like most: One OpenAI-compatible API across providers lets teams switch or add models without rewriting application code. Their most common complaint: The codebase and dependency tree are heavy, and the proxy can use gigabytes of memory.

Pros

  • One OpenAI-compatible API across providers lets teams switch or add models without rewriting application code. PHHN
  • Fallbacks, routing and model aliases let a single endpoint swap backends, including local models with cloud fallback. PHPH 2HNHN 2
  • Per-customer token spend and cost tracking is built in and is a main reason teams deploy it. PHHNHN 2
  • Open source and self-hostable, so teams avoid lock-in to a hosted gateway. PHPH 2HNHN 2
Product Hunt review tags
support for custom models2

Cons and watch-outs

  • The codebase and dependency tree are heavy, and the proxy can use gigabytes of memory. HNHN 2HN 3HN 4
  • The proxy adds noticeable latency compared with lighter Go or Rust gateways. HNHN 2HN 3HN 4
  • A supply-chain compromise of published LiteLLM versions exposed credentials in many CI pipelines. HNHN 2HN 3HN 4
  • It is buggy and not trivial to operate, with Python dependency management adding friction. HNHN 2HN 3HN 4

Themes that recur in makers' words and Hacker News comments, each linked to what it summarises, with how Product Hunt tags its reviews.

5.0★ from 24 reviews on Product Hunt ↗

What Hacker News says about LiteLLM

On Hacker News, 17% of comments about LiteLLM recommend it and 13% criticize it: net praise +4%, the least liked of 12 tools HN discusses for LLM API.

By year

YearComments choosing it, per monthNet praise
20230.1—
20240.3+50%
20252.1+7%
2026 (8 mo)2.6−7%

In their words

  • “have you tried self-hosting LiteLLM? It's what I use for the BYOK option on my service.” HN, May 2026
  • “LiteLLM's docs were simple and everything worked as expected.” HN, Aug 2025
  • “LiteLLM was incredible flaky and kept causing outages” HN, Mar 2026

Hacker News comments January 2023 to August 2026 that name LiteLLM in a sentence like "I use" or "we moved to", Claude read a sample of 60 and the counts scale it to all of them; net praise is comments recommending it minus comments criticizing it, as a share of all comments about it. How it's counted, and how HN compares with makers.

Reliability and open issues

The most wanted open issue on GitHub has 137 👍.

Most wanted on GitHub

Open on 2026-10-04 in BerriAI/litellm, with activity in the last year — issues and feature requests by 👍.

Who uses it

21 makers' products, each linked to the source that shows it, and 139 open-source projects that declare it in their code. The best known on Product Hunt: Me.bot (5,358 followers), Dashworks (1,750 followers), SellerClaw (1,008 followers), PandaProbe (887 followers), Truva (840 followers). 1 of the makers' products is gone now (domain lapsed, up for sale or taken down); it chose LiteLLM while live.

The maker says so 21Declared in code 139How evidence is collected →
Budibase
BudibaseOpen-source low-code platform for internal apps, forms and approval workflows on top of…

“We needed a simple way to work across multiple LLM providers without rebuilding integrations each time. LiteLLM gave us a clean abstraction layer, flexibility across models, and let us move faster without vendor lock-in.”

LLM APIMaker says so · source ↗
PingPrompt
PingPromptOrganize prompts, track changes, and iterate faster.

“It simplified the AI implementation and enabled support for 100+ LLMs without dealing with multiple provider APIs.”

LLM APIMaker says so · source ↗
Crossnode
CrossnodeVibe code AI agents and put them behind a payment wall

“We use LiteLLM mainly so we’re not tied to a single model provider. Makes it super easy to switch models or add fallbacks when running AI services for clients.”

LLM APIMaker says so · source ↗
Strix
StrixOpen-source AI hackers for your apps

“Big fan of liteLLM: one API for OpenAI/Anthropic/Groq/etc. Makes multi-model stacks painless”

LLM APIMaker says so · source ↗
CamelAI
CamelAIAI Data Analyst - Chat with your data

“liteLLM is a must for working with different models. We use different models for different tasks and subtasks. With liteLLM the code stays exactly the same and we can just focus on choosing the right prompts and models for the task.”

LLM APIMaker says so · source ↗
DeepTagger
DeepTaggerFrom Documents to Structured Data with Interactive Labelling

“Great library for unifying LLM access across providers, dev/test friendly!”

LLM APIMaker says so · source ↗
ARBR
ARBRControl Every AI Request

“We chose LiteLLM for its unified interface across model providers and open-source flexibility. It reduces the need to maintain separate provider integrations, letting us focus on ARBR’s evaluation, governance, and human-approved model changes. Its OpenAI-compatible interface also makes it a practical fit for teams that want to add ARBR without replacing their existing gateway.”

LLM APIMaker says so · source ↗
Dashworks
DashworksAI that answers all your team’s questions

“LiteLLM helped us quickly add support for LLMs like GPT-4o, Claude Sonnet, Gemini Pro, and Meta Llama.”

LLM APIMaker says so · source ↗
PandaProbe
PandaProbeopen source agent engineering platform

“LiteLLM is amazing and makes managing LLM clients incredibly easy. highly recommended for teams integrating multiple LLM providers in their system.”

LLM APIMaker says so · source ↗
PandaProbe Cloud
PandaProbe CloudAgent Engineering, Fully Managed.

“LiteLLM is amazing and makes managing LLM clients incredibly easy. highly recommended for teams integrating multiple LLM providers in their system.”

LLM APIMaker says so · source ↗
SellerClaw
SellerClawA team of AI agents that runs your stores across channels

“One consistent interface across every provider, so we can swap or fall back between models without rewriting code. Open-source, self-hostable, and zero vendor lock-in, plus built-in routing, retries, and cost/usage tracking out of the box.”

LLM APIMaker says so · source ↗
Athina AI
Athina AIPlatform for monitoring LLM apps in production and evaluating them, with automatic hallu…

“LiteLLM has made it super easy to switch between different models, and support custom models!”

LLM APIMaker says so · source ↗
Qodo
QodoAI code review for pull requests, the IDE and the terminal, using context from the whole…

“Easily managing API calls to multiple providers in a single unified format - very useful.”

LLM APIMaker says so · source ↗
Zro
ZroPrivate inference for coding agents

“We chose LiteLLM because it provides a unified API across model providers, making it easy to switch between models, add new providers, and manage routing without changing our application code. It significantly reduced integration complexity and sped up development.”

LLM APIMaker says so · source ↗

Open source: a project that declares LiteLLM as a dependency in its public code — verifiable, but not necessarily a live product.

What its projects tell coding agents

61 of the 101 open-source products built with LiteLLM that were worked on in the last 180 days keep instruction files for coding agents — 60%, against 53% of all the projects we read. AGENTS.md is in 47, CLAUDE.md in 33.

1,115words in the median file
uvthe package manager most named, in 27 of 51 files
75%say how to test a change
Named more in these files than in other projects' — in how many of the 51:
What AGENTS.md files say about other tools →What open-source products tell their coding agents →

Alternatives to LiteLLM

All alternatives by situation →

Further reading

Questions makers ask about LiteLLM

Is LiteLLM good for production?

21 makers' products we track run on it, and 1 of them have since gone (the product, not necessarily because of LiteLLM). The watch-out makers raise most: The codebase and dependency tree are heavy, and the proxy can use gigabytes of memory.

What is LiteLLM used for?

Open-source proxy and Python SDK giving a single OpenAI-compatible API to call 100+ LLM providers, with routing, retries and spend tracking. Makers' products we track use it for LLM API (21).

What are the pros and cons of LiteLLM?

Pros makers name: One OpenAI-compatible API across providers lets teams switch or add models without rewriting application code. Fallbacks, routing and model aliases let a single endpoint swap backends, including local models with cloud fallback. Per-customer token spend and cost tracking is built in and is a main reason teams deploy it. Cons: The codebase and dependency tree are heavy, and the proxy can use gigabytes of memory. The proxy adds noticeable latency compared with lighter Go or Rust gateways. A supply-chain compromise of published LiteLLM versions exposed credentials in many CI pipelines.

What are the alternatives to LiteLLM?

For LLM API, the other options makers' products we track chose most are OpenAI API (736), Claude (252), Gemini API (99), Groq (39), Azure OpenAI (22). Each alternative, with the situation it fits, is on the LiteLLM alternatives page.

Is LiteLLM still popular with new products?

More than before: among the 263 makers' products launched in 2026 that made the LLM API choice, 3% chose LiteLLM, against 1% of the 319 launched in 2024 or earlier.

When should you use LiteLLM?

You want that routing and spend control but self-hosted, with keys and logs on your servers (LLM API). Provider keys and request data must stay on your own servers (LLM gateway & routing). Something else fits better if: you want the strongest general models and the widest ecosystem of examples and integrations, pick OpenAI API or Claude or Gemini API; Token cost dominates your budget, for example high-volume batch or agent workloads, pick DeepSeek or Groq or Mistral AI.

Does any of my data reach LiteLLM's servers when I self-host?

No. LiteLLM says it stores no data and runs no telemetry for self-hosted instances; prompts, responses and keys stay in your infrastructure. source ↗

Do I need a database?

For virtual keys, the Admin UI and spend tracking, yes; it uses Postgres (Supabase or Neon work). Budgets are enforced from spend stored in the database, so they cap nothing on a database-less deployment. source ↗

Can I call it from Node.js or other languages?

Yes, through the gateway. Any OpenAI or Anthropic SDK works against it by changing the base URL; only the library itself is Python. source ↗

How do fallbacks work?

If a call still fails after the configured retries, LiteLLM sends it to another model group you listed as a fallback, which can be a different provider. Clients can also pass fallbacks per request. source ↗

Can I send request logs to Langfuse or LangSmith?

Yes. The gateway can log inputs, outputs and errors to Langfuse, LangSmith, OpenTelemetry, cloud storage buckets, Datadog, custom callbacks and more. source ↗

What does the enterprise license add over the open-source version?

SSO is free for up to five users; beyond that, and for SCIM, audit logs, organization-level admin roles, IP allowlists, per-team logging destinations and multi-region deployment, you need the license. source ↗

How should I deploy it in production?

On Kubernetes with the Helm chart (EKS, GKE or AKS), or with the official Terraform modules on AWS and GCP. It can run as one monolithic image or split into gateway, backend and UI services. source ↗

Is LiteLLM SOC 2 audited?

Yes, SOC 2 Type II; the current report is available through the LiteLLM Trust Center. source ↗

Is LiteLLM free?

Yes — there is a free tier a small product can run on; paid use starts at Contact sales. source ↗

Is LiteLLM open source or self-hostable?

Open source, and you can self-host it. source ↗

Can AI coding agents work with LiteLLM?

It serves an llms.txt docs index; it has an official MCP server (io.github.BerriAI/litellm-mcp).

Who uses LiteLLM?

21 makers' products we track, each with a source, and 139 open-source projects declare it in their code. The best known of them on Product Hunt: Me.bot (5,358 followers), Dashworks (1,750 followers), SellerClaw (1,008 followers), PandaProbe (887 followers), Truva (840 followers). source ↗