LiteLLM

One API for 100+ LLM providers

LLM / RAG frameworkLLM gateway / SDKMITCloud-optionalPythonBeginner

OSAI Pulse ⓘ ★★★★★★★★★★ /100 signals tracked

🐳 Docker pulls 📦 PyPI downloads / month 📦 npm downloads / month 🚀 Latest release ·

Visit website →View on GitHub

What is LiteLLM?

LiteLLM is a gateway and SDK that exposes 100+ LLM providers behind the OpenAI format, adding routing, fallbacks, budgets and observability.

Why people choose LiteLLM

OpenAI-format access to 100+ providers
Routing, fallbacks, budgets and rate limits
Proxy server for org-wide governance

LiteLLM at a glance

Category	LLM / RAG framework
Type	LLM gateway / SDK
License	MIT
Runs locally	Cloud-optional
Built with	Python
Skill level	Beginner
Best for	teams standardizing on one LLM interface

Open-source alternatives to LiteLLM

Other open-source llm / rag framework tools worth comparing:

LangChainCompose chains, tools and agents

LlamaIndexThe data framework for RAG

HaystackProduction pipelines for search and RAG

DSPyProgram — not prompt — language models

txtaiAll-in-one embeddings database

RAGFlowDeep-document-understanding RAG

Semantic KernelMicrosoft's enterprise agent framework

GraphRAGRAG that builds a knowledge graph first

InstructorReliable structured outputs from LLMs

OutlinesGuarantee valid JSON and grammars

GuidanceInterleave control and generation

LLMWareEnterprise RAG with small specialised models

FlashRankUltra-light reranking for better RAG

Sentence TransformersThe standard way to make embeddings

RagasMeasure whether your RAG is any good

PromptfooUnit tests for your prompts

LangfuseSee what your LLM app actually did

PhoenixTrace, evaluate and debug LLM apps

LiteLLM head-to-head

LiteLLM vs LangChain LiteLLM vs LlamaIndex LiteLLM vs Haystack LiteLLM vs DSPy LiteLLM vs txtai LiteLLM vs RAGFlow LiteLLM vs Semantic Kernel LiteLLM vs GraphRAG LiteLLM vs Instructor LiteLLM vs Outlines LiteLLM vs Guidance LiteLLM vs LLMWare LiteLLM vs FlashRank LiteLLM vs Sentence Transformers LiteLLM vs Ragas LiteLLM vs Promptfoo LiteLLM vs Langfuse LiteLLM vs Phoenix

FAQ

Is LiteLLM free?

LiteLLM is free and open-source (MIT license), so you can use, self-host and modify it at no cost.

Can I run LiteLLM locally?

LiteLLM can be used locally or in the cloud depending on your setup.

What is the best alternative to LiteLLM?

Popular open-source alternatives include LangChain, LlamaIndex, Haystack. See the comparisons above to choose.

Explore more open-source AI

Browse the full directory of open-source AI tools, models and projects — updated daily.

Browse all tools →