mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-15 23:31:29 +00:00
2cf209f265
* fix(mistral-ocr): forward the complete public parameter contract The Rust OCR path already lists id in the supported Mistral parameters, but litellm/ocr/main.py ran filter_out_litellm_params before handing optional_params to the bridge. That generic filter drops every key in all_litellm_params, which includes id, so the supported public Mistral OCR field id was silently stripped and never reached the provider while the rest of the contract went through. Restore id after the generic filter via an OCR-specific reserved set, keeping genuine internal params filtered. Add Rust core and ai-gateway contract tests that pin the exact serialized request body for the full supported contract and prove internal canaries are dropped, a Python regression test that id survives while internal params do not, and a deterministic capture E2E plus a live Mistral param test. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(mistral-ocr): drop unnecessary comments from parity tests Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(mistral-ocr): only forward reserved id when set; harden capture e2e Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * refactor(mistral-ocr): typed queue-injected capture harness for e2e Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(mistral-ocr): fully type E2E capture models and move gateway ocr tests Replace the remaining Any/coarse object wire types in the OCR E2E module with concrete frozen Pydantic models for the supported-param contract, internal canaries, upstream capture request, and normalized response, and build every request payload immutably per call. The capture fixture now cleans up on every failure path (terminate then always wait, kill then always wait, shutdown plus server_close, join the server thread) and requires an HTTP 200 liveness probe. The live Mistral test now exercises the complete public parameter contract with valid json_schema annotation formats. Move the ai-gateway OCR test module out of io/ocr.rs into a focused io/ocr/tests.rs. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(mistral-ocr): drop reserved id upstream when null and type the E2E harness Extend map_ocr_params so a null-valued reserved public id is omitted from the serialized request, matching the Python OCR boundary for the standalone Axum path where a caller can submit JSON id:null; add core map and serialized-body tests for both the absent and null id cases. Extract the deterministic capture harness into a typed ocr_capture_proxy helper module with an exception-safe fixture that cleans up on every failure path from capture-server creation onward (terminate then always wait, kill then always wait, shutdown plus server_close, join the server thread, close logs after the process exits) and requires an HTTP 200 liveness probe. Serialize wire payloads with by_alias so the annotation schema/additionalProperties keys match the provider contract. Replace the lambda internal canary in the Python bridge regression with a serializable value and cover the litellm_session_id and tags internal fields. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| .cargo | ||
| crates | ||
| .gitignore | ||
| ADDING_A_PROVIDER.md | ||
| AGENTS.md | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CLAUDE.md | ||
| README.md | ||
LiteLLM Rust
This workspace contains the staged Rust implementation for LiteLLM.
Rust starts as a pure transform core used by the existing Python host. Python continues to own auth, configuration, network I/O, retries, routing, logging, callbacks, spend tracking, and customer plugins until each Rust path has parity coverage and production evidence.
Crates
| Crate | Role | Pure / I/O |
|---|---|---|
| litellm-core | Translation layer — types, route contracts (traits), provider transforms (modules under providers/), and the router. Builds requests/responses; no network. | Pure |
| litellm-ai-gateway | Routes + host — the only crate that touches the network. HTTP/WebSocket I/O (modules under io/) plus the axum server binary (behind the server feature). |
I/O |
| litellm-python-bridge | PyO3 cdylib exposing Rust to the litellm Python SDK — a thin adapter over litellm-ai-gateway's I/O. | Binding |
Dependency direction (acyclic): litellm-core ← litellm-ai-gateway ← litellm-python-bridge.
Layout
crates/
core/ Route contracts, shared pure types, errors, and templates.
src/ocr/
providers/ Provider-specific pure transforms.
src/mistral/ocr/transformation.rs
python-bridge/ PyO3 bridge for Python LiteLLM.
The folder shape should follow the Python provider tree:
providers/src/<provider>/<route>/transformation.rs. The bridge should expose
one function per top-level route, starting with ocr(payload).
Checks
Run these before pushing Rust changes. GitHub Actions runs the same checks for
changes under litellm-rust/.
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace