diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 6894c4660f7..a86c7c71e08 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,4 +1,4 @@ -# LiteLLM Proxy Architecture +# LiteLLM Architecture - LiteLLM SDK + AI Gateway This document helps contributors understand where to make changes in the LiteLLM proxy. @@ -65,6 +65,53 @@ graph TD Main --> Client ``` +### SDK Components (litellm/) + +```mermaid +graph TD + subgraph "SDK Entry Points" + Completion["litellm.completion()"] + Messages["litellm.messages()"] + end + + subgraph "main.py" + Main["completion()
acompletion()"] + end + + subgraph "utils.py" + GetProvider["get_llm_provider()"] + end + + subgraph "llms/custom_httpx/" + Handler["llm_http_handler.py
BaseLLMHTTPHandler"] + HTTP["http_handler.py
HTTPHandler / AsyncHTTPHandler"] + end + + subgraph "llms/{provider}/chat/" + Transform["transformation.py
ProviderConfig"] + end + + subgraph "litellm_core_utils/" + Streaming["streaming_handler.py"] + TokenCounter["token_counter.py"] + end + + subgraph "integrations/" + Callbacks["custom_logger.py
Langfuse, Datadog, etc."] + end + + Completion --> Main + Messages --> Main + Main --> GetProvider + GetProvider --> Handler + Handler --> Transform + Transform --> HTTP + HTTP --> Provider["LLM Provider API"] + Handler --> Streaming + Handler --> Callbacks + Main --> TokenCounter +``` + **Key files:** - `proxy/proxy_server.py` - Main API endpoints - `proxy/auth/` - Authentication