From e55b9f981a4f382ea684a781f2f3710d96ac1af8 Mon Sep 17 00:00:00 2001 From: Ishaan Jaff Date: Wed, 24 Jun 2026 15:58:24 -0700 Subject: [PATCH] docs: note Tokio route contract --- litellm-rust/ADDING_A_PROVIDER.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/litellm-rust/ADDING_A_PROVIDER.md b/litellm-rust/ADDING_A_PROVIDER.md index dc4b95ea173..4a85af61905 100644 --- a/litellm-rust/ADDING_A_PROVIDER.md +++ b/litellm-rust/ADDING_A_PROVIDER.md @@ -4,5 +4,5 @@ - Put provider-specific transforms in `crates/providers/src///transformation.rs`, mirroring the Python provider tree and exposing a `const __CONFIG`. - The provider config owns three pure steps: map LiteLLM params, transform the LiteLLM request into the provider request, and transform the provider response back into the LiteLLM response. - If the provider has a reverse or normalization step, keep it pure and explicit next to the transforms; do not hide reverse mapping inside the HTTP transport. -- Route host functions in `crates/providers/src/.rs` must be async: resolve auth/base URL, call the transforms, send with async transport, then call the response transform. +- Route host functions in `crates/providers/src/.rs` must be async: resolve auth/base URL, call the transforms, send with async transport, then call the response transform. Use Tokio/async all the way through Rust route I/O; only the PyO3 sync compatibility wrapper should `block_on` the async route, and it must release the GIL while waiting. - Register modules in `lib.rs` / `mod.rs`, add parity tests for params/request/response behavior, then run `cargo fmt && cargo clippy --workspace --all-targets --locked -- -D warnings && cargo test --workspace --locked`.