litellm
LiteLLM is an open-source AI gateway and Python SDK designed to simplify interactions with over 100 large language model providers including OpenAI, Anthropic, Google Vertex AI, Amazon Bedrock, Azure, Cohere, Hugging Face, Sagemaker, and NVIDIA NIM. It provides a unified interface that standardizes all API calls into the OpenAI format, allowing developers to swap models or providers without rewriting application code. The suite offers two primary deployment modes: a lightweight Python SDK for direct library integration and a self-hosted or cloud-ready Proxy Server that functions as a centralized enterprise gateway. Key features include cost tracking and spend management, virtual key management for security, input and output guardrails for safety, automatic load balancing across multiple models, and comprehensive request logging. The system is built for production use with high performance benchmarks demonstrating P95 latencies under 10 milliseconds at 1000 requests per second. It supports virtual keys for gra