LiteLLM
CategoryOpen-source LLM gateway
PricingOpen source (MIT); enterprise tier
Verdict
The default self-hosted gateway: one OpenAI-compatible proxy across 100+ providers with budgets, caching, routing, fallbacks, and per-key cost tracking. Engineering-led teams should start here.
Strengths
- Self-hosted and free; no traffic leaves your infra
- Budgets/rate limits per API key, team, or user — runaway-spend protection built in
- Supports provider prompt caching, batch routing, and fallback chains
Limitations
- You operate it — upgrades, scaling, and config are on you
- UI is functional, not polished; observability is thinner than dedicated tools