mirror of https://github.com/BerriAI/litellm.git synced 2026-10-10 03:28:53 +00:00

History

Devin AI 7f526e0b08 Some checks failed LiteLLM Rust / rustfmt, clippy, test (push) Has been cancelled Details refactor(rust-gateway): harden OCR transport error contract and validation Generic fixed messages for all JSON/body parse failures so serde field, tag, and value text can never cross the Axum response, with unit and route marker tests covering query, base64, credential-path, and unknown-document-type inputs. Map every CoreError through a typed Serialize error envelope with an exhaustive match instead of serde_json::Value, dropping error.to_string() for the wrapping variants and preserving valid upstream 4xx/5xx status codes. Normalize declared upload MIME values case-insensitively before image classification and data-URI construction, and treat generic values with parameters as generic so magic-byte sniffing still runs. Reject a multipart document form field as reserved and reject blank JSON and multipart model before routing, keeping duplicate file, model, and timeout rejection. Move the transport unit tests into transport/tests.rs, and join or abort the gateway and mock upstream handles in route tests while draining request bodies before mock responses. Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>		2026-07-17 00:53:18 +00:00
..
.cargo	refactor(litellm-rust): dissolve providers into core + ai-gateway (strict 3-crate layers) (#31218 )	2026-06-24 12:22:43 -07:00
crates	refactor(rust-gateway): harden OCR transport error contract and validation	2026-07-17 00:53:18 +00:00
.gitignore	feat: add LiteLLM Rust workspace with Mistral OCR bridge (#31033 )	2026-06-23 13:16:47 -07:00
ADDING_A_PROVIDER.md	feat: add openai realtime translation layer to litellm-rust (1/2) (#31129 )	2026-06-23 15:48:51 -07:00
AGENTS.md	refactor(litellm-rust): dissolve providers into core + ai-gateway (strict 3-crate layers) (#31218 )	2026-06-24 12:22:43 -07:00
Cargo.lock	feat(rust-gateway): expose OCR over Axum	2026-07-16 22:13:39 +00:00
Cargo.toml	feat(ocr): route all OCR through Rust	2026-06-25 14:16:42 -07:00
CLAUDE.md	chore(rust-gateway): drop out-of-scope skill and CLAUDE.md policy changes	2026-07-16 23:39:29 +00:00
README.md	refactor(litellm-rust): dissolve providers into core + ai-gateway (strict 3-crate layers) (#31218 )	2026-06-24 12:22:43 -07:00

README.md

LiteLLM Rust

This workspace contains the staged Rust implementation for LiteLLM.

Rust starts as a pure transform core used by the existing Python host. Python continues to own auth, configuration, network I/O, retries, routing, logging, callbacks, spend tracking, and customer plugins until each Rust path has parity coverage and production evidence.

Crates

Crate	Role	Pure / I/O
litellm-core	Translation layer — types, route contracts (traits), provider transforms (modules under providers/), and the router. Builds requests/responses; no network.	Pure
litellm-ai-gateway	Routes + host — the only crate that touches the network. HTTP/WebSocket I/O (modules under io/) plus the axum server binary (behind the `server` feature).	I/O
litellm-python-bridge	PyO3 cdylib exposing Rust to the litellm Python SDK — a thin adapter over litellm-ai-gateway's I/O.	Binding

Dependency direction (acyclic): litellm-core ← litellm-ai-gateway ← litellm-python-bridge.

Layout

crates/
  core/           Route contracts, shared pure types, errors, and templates.
    src/ocr/
  providers/      Provider-specific pure transforms.
    src/mistral/ocr/transformation.rs
  python-bridge/  PyO3 bridge for Python LiteLLM.

The folder shape should follow the Python provider tree: providers/src/<provider>/<route>/transformation.rs. The bridge should expose one function per top-level route, starting with ocr(payload).

Checks

Run these before pushing Rust changes. GitHub Actions runs the same checks for changes under litellm-rust/.

cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace