mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-06 08:16:43 +00:00
0b8817afbb
* feat(bedrock): add audio transcription via Converse with py->rust bridge Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(ci): exclude rust transcription rollout flag from docs check Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(bedrock): await rust/python fallback in async transcription dispatch Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(bedrock): cover audio transcription rust dispatch Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * refactor(bedrock): route audio transcription through rust Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * refactor(bedrock): move rust transcription dispatch out of main Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(bedrock): include rust transcription coverage shard Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
1.7 KiB
1.7 KiB
CLAUDE.md
Rules for litellm-rust/crates/python-bridge.
Responsibility
python-bridge is the PyO3 boundary between Python LiteLLM and Rust transforms.
Keep this crate thin. It adapts Python objects to Rust payloads and returns
Python-compatible dictionaries.
Bridge Shape
- Prefer one stable method per top-level LiteLLM route, for example
ocr(payload). - Do not add one exported PyO3 function per provider helper unless there is a measured reason.
- Provider dispatch belongs in Rust route modules such as
litellm_providers::ocr, not in this PyO3 crate. - Python owns rollout state and fallback. Rust should return errors; Python decides whether to raise or fall back. For a rust-only provider/route (no Python reference), the Python side is a thin dispatch that calls Rust and raises when the bridge is unavailable, with no fallback.
- Keep the Python interface minimal (well under 100 lines per route): it only
marshals inputs and calls Rust. Do not add per-route feature flags, and do
not put provider dispatch in
litellm/main.py; it lives in a thin dispatch class underlitellm/llms/<provider>/<route>/.
Data Handling
- OCR payloads can contain personal data and large base64 images. Do not log payloads or provider responses.
- Avoid copying large payloads more than needed. The current JSON round-trip is acceptable for the first scaffold, but future performance work should evaluate direct PyO3 conversion before expanding Rust coverage to image-heavy paths.
- Do not expose raw Rust errors that include document contents or upstream bodies.
Tests
cargo test --workspacemust compile this crate.- Python tests must cover bridge disabled, bridge enabled, and module-missing fallback behavior for every exposed route.