scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
litellm/litellm-rust/crates/gateway-inference
History
Exact
Yujong Lee 9a08f212c8 refactor(rust): replace ai gateway with inference crate
2026-09-04 14:57:48 -07:00
..
benchmarks/realtime refactor(rust): replace ai gateway with inference crate 2026-09-04 14:57:48 -07:00
src refactor(rust): replace ai gateway with inference crate 2026-09-04 14:57:48 -07:00
tests refactor(rust): replace ai gateway with inference crate 2026-09-04 14:57:48 -07:00
AGENTS.md refactor(rust): replace ai gateway with inference crate 2026-09-04 14:57:48 -07:00
ARCHITECTURE.md refactor(rust): replace ai gateway with inference crate 2026-09-04 14:57:48 -07:00
Cargo.toml refactor(rust): replace ai gateway with inference crate 2026-09-04 14:57:48 -07:00
README.md refactor(rust): replace ai gateway with inference crate 2026-09-04 14:57:48 -07:00

README.md

LiteLLM Gateway Inference

litellm-gateway-inference is the reusable, framework-independent inference domain used by the Rust gateway server and Python bridge

It owns inference integrations, transport-neutral orchestration, and legacy OCR, audio transcription, and realtime I/O that have not yet moved into litellm-core. It has no Axum or Tower dependency

The executable server, HTTP/WebSocket routes, auth extractors, application state, config startup, Docker image, and deployment files live in ../gateway-server