LiteLLM

Open-source proxy and Python SDK giving a single OpenAI-compatible API to call 100+ LLM providers, with routing, retries and spend tracking.

Works with AI agents:llms.txtMCP
Ask your AI about this, with this page as the source:ChatGPT ↗Claude ↗Perplexity ↗

LiteLLM, from BerriAI, is an MIT-licensed project with two parts: a Python library that calls 100+ LLM providers through one function and always returns OpenAI-shaped responses, and an AI gateway (proxy) you run yourself that offers the same thing as an OpenAI-compatible HTTP API for any language.

With the library you call litellm.completion(model="provider/model", ...), and its Router adds retries, load balancing and fallbacks between named model groups. The gateway is a Docker image: you register models and provider credentials in a config.yaml or the built-in Admin UI, then hand out virtual keys to apps and teammates. Postgres holds models, keys and spend logs, and the gateway needs a master key and a salt key that encrypts stored credentials. Clients such as the OpenAI or Anthropic SDKs, Claude Code or Codex connect by changing their base URL.

Useful for a small team: budgets and tpm/rpm limits per key, user or team; automatic failover to another provider after retries; logging to Langfuse, LangSmith, OpenTelemetry, S3 or Datadog; and no telemetry sent to LiteLLM when self-hosted. It runs with Docker Compose, one-click Railway or Render templates, Helm on EKS, GKE or AKS, or Terraform modules for AWS and GCP.

The main limit is that you operate it: a database, upgrades, and a salt key that cannot be rotated in place. Budgets are not enforced without the database, and SSO beyond five users, audit logs and multi-region deployment need an enterprise license.

Where it fits

How LiteLLM itself is built

18 tools, from its own code, website and Product Hunt page.

Who uses it

21 makers' products, each linked to the source that shows it, and 97 open-source projects that declare it in their code.

The maker says so 21Declared in code 97How evidence is collected →
BudibaseOpen-source low-code platform for internal apps, forms and approval workflows on top of…

“We needed a simple way to work across multiple LLM providers without rebuilding integrations each time. LiteLLM gave us a clean abstraction layer, flexibility across models, and let us move faster without vendor lock-in.”

LLM APIMaker says so · source ↗
PingPromptOrganize prompts, track changes, and iterate faster.

“It simplified the AI implementation and enabled support for 100+ LLMs without dealing with multiple provider APIs.”

LLM APIMaker says so · source ↗
StrixOpen-source AI hackers for your apps

“Big fan of liteLLM: one API for OpenAI/Anthropic/Groq/etc. Makes multi-model stacks painless”

LLM APIMaker says so · source ↗
CrossnodeVibe code AI agents and put them behind a payment wall

“We use LiteLLM mainly so we’re not tied to a single model provider. Makes it super easy to switch models or add fallbacks when running AI services for clients.”

LLM APIMaker says so · source ↗
CamelAIAI Data Analyst - Chat with your data

“liteLLM is a must for working with different models. We use different models for different tasks and subtasks. With liteLLM the code stays exactly the same and we can just focus on choosing the right prompts and models for the task.”

LLM APIMaker says so · source ↗
PandaProbeopen source agent engineering platform

“LiteLLM is amazing and makes managing LLM clients incredibly easy. highly recommended for teams integrating multiple LLM providers in their system.”

LLM APIMaker says so · source ↗
PandaProbe CloudAgent Engineering, Fully Managed.

“LiteLLM is amazing and makes managing LLM clients incredibly easy. highly recommended for teams integrating multiple LLM providers in their system.”

LLM APIMaker says so · source ↗
ARBRControl Every AI Request

“We chose LiteLLM for its unified interface across model providers and open-source flexibility. It reduces the need to maintain separate provider integrations, letting us focus on ARBR’s evaluation, governance, and human-approved model changes. Its OpenAI-compatible interface also makes it a practical fit for teams that want to add ARBR without replacing their existing gateway.”

LLM APIMaker says so · source ↗
SellerClawA team of AI agents that runs your stores across channels

“One consistent interface across every provider, so we can swap or fall back between models without rewriting code. Open-source, self-hostable, and zero vendor lock-in, plus built-in routing, retries, and cost/usage tracking out of the box.”

LLM APIMaker says so · source ↗
Athina AIMonitor LLMs and automatically detect hallucinations in prod

“LiteLLM has made it super easy to switch between different models, and support custom models!”

LLM APIMaker says so · source ↗
DashworksAI that answers all your team’s questions

“LiteLLM helped us quickly add support for LLMs like GPT-4o, Claude Sonnet, Gemini Pro, and Meta Llama.”

LLM APIMaker says so · source ↗
ZroPrivate inference for coding agents

“We chose LiteLLM because it provides a unified API across model providers, making it easy to switch between models, add new providers, and manage routing without changing our application code. It significantly reduced integration complexity and sped up development.”

LLM APIMaker says so · source ↗

Open source: a project that declares LiteLLM as a dependency in its public code — verifiable, but not necessarily a live product.

What makers say

16 makers on why they use LiteLLM, in their own words on Product Hunt.

liteLLM is a must for working with different models. We use different models for different tasks and subtasks. With liteLLM the code stays exactly the same and we can just focus on choosing the right prompts and models for the task.
CamelAISep 2026 ↗
We use LiteLLM mainly so we’re not tied to a single model provider. Makes it super easy to switch models or add fallbacks when running AI services for clients.
CrossnodeSep 2026 ↗
LiteLLM is amazing and makes managing LLM clients incredibly easy. highly recommended for teams integrating multiple LLM providers in their system.
PandaProbeSep 2026 ↗
LiteLLM is amazing and makes managing LLM clients incredibly easy. highly recommended for teams integrating multiple LLM providers in their system.
PandaProbe CloudSep 2026 ↗
It simplified the AI implementation and enabled support for 100+ LLMs without dealing with multiple provider APIs.
PingPromptSep 2026 ↗

Loved and watch-outs

Themes that recur in makers' words and Hacker News comments, each linked to what it summarises, with how Product Hunt tags its reviews.

Most loved
  • One OpenAI-compatible API across providers lets teams switch or add models without rewriting application code. PHHN
  • Fallbacks, routing and model aliases let a single endpoint swap backends, including local models with cloud fallback. PHPH 2HNHN 2
  • Per-customer token spend and cost tracking is built in and is a main reason teams deploy it. PHHNHN 2
  • Open source and self-hostable, so teams avoid lock-in to a hosted gateway. PHPH 2HNHN 2
Watch-outs
  • The codebase and dependency tree are heavy, and the proxy can use gigabytes of memory. HNHN 2HN 3HN 4
  • The proxy adds noticeable latency compared with lighter Go or Rust gateways. HNHN 2HN 3HN 4
  • A supply-chain compromise of published LiteLLM versions exposed credentials in many CI pipelines. HNHN 2HN 3HN 4
  • It is buggy and not trivial to operate, with Python dependency management adding friction. HNHN 2HN 3HN 4
On Product Hunt 5.0★ · 24 reviews
support for custom models 2
Read the reviews on Product Hunt ↗

Reliability and open issues

Most wanted on GitHubOpen on 2026-10-04 in BerriAI/litellm, with activity in the last year — issues and feature requests by 👍.

Alternatives to LiteLLM

All alternatives by situation →

On Product Hunt, people weigh it against: Eden AI, OpenRouter, Groq.

Questions makers ask about LiteLLM

Does any of my data reach LiteLLM's servers when I self-host?

No. LiteLLM says it stores no data and runs no telemetry for self-hosted instances; prompts, responses and keys stay in your infrastructure. source ↗

Do I need a database?

For virtual keys, the Admin UI and spend tracking, yes; it uses Postgres (Supabase or Neon work). Budgets are enforced from spend stored in the database, so they cap nothing on a database-less deployment. source ↗

Can I call it from Node.js or other languages?

Yes, through the gateway. Any OpenAI or Anthropic SDK works against it by changing the base URL; only the library itself is Python. source ↗

How do fallbacks work?

If a call still fails after the configured retries, LiteLLM sends it to another model group you listed as a fallback, which can be a different provider. Clients can also pass fallbacks per request. source ↗

Can I send request logs to Langfuse or LangSmith?

Yes. The gateway can log inputs, outputs and errors to Langfuse, LangSmith, OpenTelemetry, cloud storage buckets, Datadog, custom callbacks and more. source ↗

What does the enterprise license add over the open-source version?

SSO is free for up to five users; beyond that, and for SCIM, audit logs, organization-level admin roles, IP allowlists, per-team logging destinations and multi-region deployment, you need the license. source ↗

How should I deploy it in production?

On Kubernetes with the Helm chart (EKS, GKE or AKS), or with the official Terraform modules on AWS and GCP. It can run as one monolithic image or split into gateway, backend and UI services. source ↗

Is LiteLLM SOC 2 audited?

Yes, SOC 2 Type II; the current report is available through the LiteLLM Trust Center. source ↗

Is LiteLLM free?

Yes — there is a free tier a small product can run on; paid use starts at Contact sales. source ↗

Is LiteLLM open source or self-hostable?

Open source, and you can self-host it. source ↗

Can AI coding agents work with LiteLLM?

It serves an llms.txt docs index; it has an official MCP server (io.github.BerriAI/litellm-mcp).

Who uses LiteLLM?

21 makers' products we track, each with a source, and 97 open-source projects declare it in their code. source ↗