diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 7b4f15e8c80..de2455a51a0 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -4,6 +4,21 @@ This document helps contributors understand where to make changes in LiteLLM. --- +## How It Works + +The AI Gateway (Proxy) uses the LiteLLM SDK internally for all LLM calls: + +``` +OpenAI SDK (client) ──▶ AI Gateway (proxy/) ──▶ LiteLLM SDK (litellm/) ──▶ LLM API +Anthropic SDK (client) ──▶ AI Gateway (proxy/) ──▶ LiteLLM SDK (litellm/) ──▶ LLM API +Any HTTP client ──▶ AI Gateway (proxy/) ──▶ LiteLLM SDK (litellm/) ──▶ LLM API +``` + +The **AI Gateway** adds authentication, rate limiting, budgets, and routing on top of the SDK. +The **SDK** handles the actual LLM provider calls, request/response transformations, and streaming. + +--- + ## 1. AI Gateway (Proxy) Request Flow The AI Gateway (`litellm/proxy/`) wraps the SDK with authentication, rate limiting, and management features.