litellm/litellm-rust/crates/python-bridge/CLAUDE.md

CLAUDE.md

Rules for litellm-rust/crates/python-bridge.

Responsibility

python-bridge is the PyO3 boundary between Python LiteLLM and Rust transforms. Keep this crate thin. It exposes LiteLLM Rust APIs, assembles domain requests, maps domain errors to Python exceptions, and delegates generic conversion and GIL handling to litellm-python-interop.

Bridge Shape

• Prefer one stable method per top-level LiteLLM route, for example messages(...), calling the matching litellm-core entrypoint.
• Do not add one exported PyO3 function per provider helper unless there is a measured reason.
• Provider dispatch belongs in the litellm-core route module (e.g. litellm_core::messages), not in this PyO3 crate.
• Python owns rollout state and fallback. Rust should return errors; Python decides whether to raise or fall back. For a rust-only provider/route (no Python reference), the Python side is a thin dispatch that calls Rust and raises when the bridge is unavailable, with no fallback.
• Keep the Python interface minimal (well under 100 lines per route): it only marshals inputs and calls Rust. Do not add per-route feature flags, and do not put provider dispatch in litellm/main.py; it lives in a thin dispatch class under litellm/llms/<provider>/<route>/.

Data Handling

• OCR payloads can contain personal data and large base64 images. Do not log payloads or provider responses.
• Avoid copying large payloads more than needed. The current JSON round-trip is acceptable for the first scaffold, but future performance work should evaluate direct PyO3 conversion before expanding Rust coverage to image-heavy paths.
• Do not expose raw Rust errors that include document contents or upstream bodies.

Tests

• cargo test --workspace must compile this crate.
• Python tests must cover bridge disabled, bridge enabled, and module-missing fallback behavior for every exposed route.