mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-14 23:21:35 +00:00
docs: bound rust OCR HTTP exception
This commit is contained in:
parent
82ec9aeb70
commit
5d5dd89829
2 changed files with 6 additions and 0 deletions
|
|
@ -5,4 +5,5 @@
|
|||
- The provider config owns three pure steps: map LiteLLM params, transform the LiteLLM request into the provider request, and transform the provider response back into the LiteLLM response.
|
||||
- If the provider has a reverse or normalization step, keep it pure and explicit next to the transforms; do not hide reverse mapping inside the HTTP transport.
|
||||
- Route host functions in `crates/providers/src/<route>.rs` must be async: resolve auth/base URL, call the transforms, send with async transport, then call the response transform. Use Tokio/async all the way through Rust route I/O; only the PyO3 sync compatibility wrapper should `block_on` the async route, and it must release the GIL while waiting.
|
||||
- Do not add per-provider HTTP clients casually. Today Rust cannot call Python's `BaseLLMHTTPHandler`; if a route needs end-to-end Rust I/O, keep the async transport route-scoped, opt-in from Python, and do not broaden it to more providers until there is a shared Rust HTTP abstraction.
|
||||
- Register modules in `lib.rs` / `mod.rs`, add parity tests for params/request/response behavior, then run `cargo fmt && cargo clippy --workspace --all-targets --locked -- -D warnings && cargo test --workspace --locked`.
|
||||
|
|
|
|||
|
|
@ -28,6 +28,11 @@ const OCR_TIMEOUT_SECS: u64 = 600;
|
|||
const ERROR_BODY_MAX_CHARS: usize = 256;
|
||||
|
||||
/// Process-wide async HTTP client (connection pool + TLS reused across calls).
|
||||
///
|
||||
/// The Python fallback path uses LiteLLM's standard `BaseLLMHTTPHandler`. This
|
||||
/// Rust path is opt-in and owns end-to-end OCR I/O, so it cannot call the
|
||||
/// Python handler directly; keep this route-scoped until litellm-rust has a
|
||||
/// shared HTTP abstraction.
|
||||
fn http_client() -> &'static reqwest::Client {
|
||||
static CLIENT: OnceLock<reqwest::Client> = OnceLock::new();
|
||||
CLIENT.get_or_init(|| {
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue