scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 114
litellm/litellm-rust/crates/gateway-inference/ARCHITECTURE.md

559 B
Raw Permalink Blame History

gateway-inference architecture

litellm-gateway-inference is the first extracted gateway domain. It sits between hosts and litellm-core, exposing framework-independent inference services and callback integrations without depending on an HTTP framework

flowchart LR
  S[litellm-gateway-server] --> G[litellm-gateway-inference runtime]
  B[litellm-python-bridge] --> G
  G --> C[litellm-core]
  S --> C
  S --> F[litellm-config]

The server may depend on the runtime, core, and config crates. Reusable crates must not depend on the server