466b44e318
Adds crates/python-bridge/src/architecture.rs, a source-scan test in the spirit of workspace_crate_allowlist.rs: layering rules that currently live only in AGENTS.md become executable. Rules on production code (text before each file's trailing #[cfg(test)] module): - GIL attach/detach and block_on appear in python-bridge only in execution.rs; scattered interpreter calls are how GIL-ordering deadlocks and per-handoff contention creep in. - Tokio runtime construction appears only in execution.rs and the #[pymodule] init site in lib.rs; one shared runtime per process. - SendWrapper is banned in both PyO3 crates; it converts !Send Python values into cross-thread panics on Tokio workers. - python-interop stays domain-neutral and never blocks on futures or builds runtimes. Deletes the dead routes/runtime.rs: an undeclared byte-for-byte duplicate of execution.rs whose Python::attach/block_on usage would violate the new boundary (also deleted independently in #39577; both sides delete the same file, so the merge is trivial). AGENTS.md gains the enforcement note, mirroring the crate-allowlist convention.
3.2 KiB
AGENTS.md
litellm-rust has four crates. A crate is a layer or shared foundation, not a route. Routes (ocr, realtime, chat) and providers (mistral, openai) are modules inside the layers.
Crates
| Crate | Role |
|---|---|
| litellm-core | The LiteLLM SDK in Rust. One public entrypoint per top-level call (messages::messages()), owning types, transforms, provider resolution, auth, and the provider HTTP call. Call it, get a typed response. |
| litellm-ai-gateway | The axum server (behind the server feature) plus the WebSocket hosts. Translates HTTP/WS to core entrypoints; owns no provider logic and no handlers. |
| litellm-python-interop | Domain-neutral PyO3 foundation for GIL handling and typed Python/Serde conversion. |
| litellm-python-bridge | PyO3 cdylib exposing LiteLLM Rust APIs to the Python SDK. Owns API registration, domain wiring, and Python exception mapping. |
Dependency direction is acyclic: litellm-python-bridge depends on the domain layers and litellm-python-interop; the interop foundation depends on no LiteLLM domain crate.
Interpreter and runtime boundaries are enforced by crates/python-bridge/src/architecture.rs: GIL attach/detach and block_on live only in python-bridge/src/execution.rs, Tokio runtime construction only there and at the #[pymodule] init site in src/lib.rs, SendWrapper is banned, and python-interop stays domain-neutral. The test fails until its allowlist is updated — moving a boundary is a deliberate act that also updates the crate AGENTS.md.
Where a route lives
A top-level LiteLLM call is a module under crates/core/src/<route>/, shaped like messages:
core/src/messages/
mod.rs # pub async fn messages(..) -> CoreResult<..> (+ messages_stream for SSE)
types.rs # request/response types, MessagesRequest
transformation.rs # the provider template trait
prepare.rs # provider resolution, auth headers, URL
handler.rs # the provider call
client.rs # the shared reqwest client
Handlers never live in ai-gateway. ocr, audio_transcription, and realtime are still hosted there from before this rule; they move to core as they are touched.
Adding a crate: default to a module. A new crate requires a real trigger: separate artifact (binary/cdylib), proc-macro, shared foundation, or publishable standalone. A new provider or route is none of these.
Adding a crate fails crates/core/tests/workspace_crate_allowlist.rs until you update its allowlist and this file — intentional.
Style
All Rust in litellm-rust/ follows the official Rust Style Guide:
https://doc.rust-lang.org/style-guide/
rustfmt implements its formatting by default, so run cargo fmt before committing; CI gates every PR on cargo fmt --check. Do not hand-format against rustfmt or add a rustfmt.toml that diverges from the default style.
Beyond formatting, follow the guide's naming and idiom conventions rustfmt cannot auto-apply: snake_case items/functions/modules, UpperCamelCase types/traits/variants, SCREAMING_SNAKE_CASE constants/statics (acronyms as one word, e.g. HttpClient), and the import grouping and item ordering it prescribes. See CLAUDE.md for the detailed version.