scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-09 03:18:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 109
litellm/litellm-rust/crates/gateway-server/ARCHITECTURE.md

649 B
Raw Permalink Blame History

gateway-server architecture

The Rust gateway server is the root composition crate. It owns the Axum binary, HTTP/WebSocket routes, auth extractors, application state, and startup config. It delegates inference behavior to litellm-gateway-inference; future domain crates plug into this composition root the same way

flowchart LR
  C[client] <--> S[litellm-gateway-server<br/>Axum host]
  S --> G[litellm-gateway-inference<br/>runtime and integrations]
  G --> K[litellm-core]
  G <--> O[OpenAI realtime]
  G -. spend tracking callback .-> P[litellm proxy]
  F[litellm-config<br/>load-time only] --> S
  F -. Python backend .-> P