mirror of https://github.com/BerriAI/litellm.git synced 2026-09-08 22:21:35 +00:00

History

yuneng-jiang f3d20153b3 build(rust): raise pyo3 to 0.29 so the native bridge compiles on Python 3.14 (#33798 ) pyo3 0.23.5 hard-caps the interpreter at Python 3.13, so building the native bridge against a 3.14 interpreter aborts inside pyo3-ffi's build script before anything links. This raises pyo3 and pyo3-async-runtimes to 0.29 (currently the newest line, and the range starting at 0.26 that supports 3.14) and migrates the three call sites whose APIs were renamed across that range: Python::with_gil is now Python::attach and Python::allow_threads is now Python::detach. On a GIL-enabled interpreter those are pure renames with identical semantics, so behavior on 3.10 through 3.13 is unchanged Verified by compiling the native module for cp313 and cp314 and driving it directly on both interpreters: gil_stats reports exactly one GIL release per sync OCR call and the async path completes, matching the 0.23.5 baseline. cargo fmt, clippy, and the workspace tests pass on both 3.13 and 3.14 with the lockfile locked, and the lock churn is confined to the pyo3 crates Part of #26343; addresses the pyo3 build failure reported in #33116		2026-07-18 00:40:15 +00:00
..
crates	build(rust): raise pyo3 to 0.29 so the native bridge compiles on Python 3.14 (#33798 )	2026-07-18 00:40:15 +00:00
.gitignore	feat: add LiteLLM Rust workspace with Mistral OCR bridge (#31033 )	2026-06-23 13:16:47 -07:00
ADDING_A_PROVIDER.md	feat: add openai realtime translation layer to litellm-rust (1/2) (#31129 )	2026-06-23 15:48:51 -07:00
AGENTS.md	refactor(litellm-rust): dissolve providers into core + ai-gateway (strict 3-crate layers) (#31218 )	2026-06-24 12:22:43 -07:00
Cargo.lock	build(rust): raise pyo3 to 0.29 so the native bridge compiles on Python 3.14 (#33798 )	2026-07-18 00:40:15 +00:00
Cargo.toml	build(rust): raise pyo3 to 0.29 so the native bridge compiles on Python 3.14 (#33798 )	2026-07-18 00:40:15 +00:00
CLAUDE.md	feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks (#31134 )	2026-06-24 15:25:10 -07:00
README.md	refactor(litellm-rust): dissolve providers into core + ai-gateway (strict 3-crate layers) (#31218 )	2026-06-24 12:22:43 -07:00

README.md

LiteLLM Rust

This workspace contains the staged Rust implementation for LiteLLM.

Rust starts as a pure transform core used by the existing Python host. Python continues to own auth, configuration, network I/O, retries, routing, logging, callbacks, spend tracking, and customer plugins until each Rust path has parity coverage and production evidence.

Crates

Crate	Role	Pure / I/O
litellm-core	Translation layer — types, route contracts (traits), provider transforms (modules under providers/), and the router. Builds requests/responses; no network.	Pure
litellm-ai-gateway	Routes + host — the only crate that touches the network. HTTP/WebSocket I/O (modules under io/) plus the axum server binary (behind the `server` feature).	I/O
litellm-python-bridge	PyO3 cdylib exposing Rust to the litellm Python SDK — a thin adapter over litellm-ai-gateway's I/O.	Binding

Dependency direction (acyclic): litellm-core ← litellm-ai-gateway ← litellm-python-bridge.

Layout

crates/
  core/           Route contracts, shared pure types, errors, and templates.
    src/ocr/
  providers/      Provider-specific pure transforms.
    src/mistral/ocr/transformation.rs
  python-bridge/  PyO3 bridge for Python LiteLLM.

The folder shape should follow the Python provider tree: providers/src/<provider>/<route>/transformation.rs. The bridge should expose one function per top-level route, starting with ocr(payload).

Checks

Run these before pushing Rust changes. GitHub Actions runs the same checks for changes under litellm-rust/.

cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace