mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-17 23:51:30 +00:00
0b8817afbb
* feat(bedrock): add audio transcription via Converse with py->rust bridge Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(ci): exclude rust transcription rollout flag from docs check Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * fix(bedrock): await rust/python fallback in async transcription dispatch Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(bedrock): cover audio transcription rust dispatch Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * refactor(bedrock): route audio transcription through rust Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * refactor(bedrock): move rust transcription dispatch out of main Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> * test(bedrock): include rust transcription coverage shard Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
42 lines
1.7 KiB
Markdown
42 lines
1.7 KiB
Markdown
# CLAUDE.md
|
|
|
|
Rules for `litellm-rust/crates/python-bridge`.
|
|
|
|
## Responsibility
|
|
|
|
`python-bridge` is the PyO3 boundary between Python LiteLLM and Rust transforms.
|
|
Keep this crate thin. It adapts Python objects to Rust payloads and returns
|
|
Python-compatible dictionaries.
|
|
|
|
## Bridge Shape
|
|
|
|
- Prefer one stable method per top-level LiteLLM route, for example
|
|
`ocr(payload)`.
|
|
- Do not add one exported PyO3 function per provider helper unless there is a
|
|
measured reason.
|
|
- Provider dispatch belongs in Rust route modules such as
|
|
`litellm_providers::ocr`, not in this PyO3 crate.
|
|
- Python owns rollout state and fallback. Rust should return errors; Python
|
|
decides whether to raise or fall back. For a rust-only provider/route (no
|
|
Python reference), the Python side is a thin dispatch that calls Rust and
|
|
raises when the bridge is unavailable, with no fallback.
|
|
- Keep the Python interface minimal (well under 100 lines per route): it only
|
|
marshals inputs and calls Rust. Do not add per-route feature flags, and do
|
|
not put provider dispatch in `litellm/main.py`; it lives in a thin dispatch
|
|
class under `litellm/llms/<provider>/<route>/`.
|
|
|
|
## Data Handling
|
|
|
|
- OCR payloads can contain personal data and large base64 images. Do not log
|
|
payloads or provider responses.
|
|
- Avoid copying large payloads more than needed. The current JSON round-trip is
|
|
acceptable for the first scaffold, but future performance work should evaluate
|
|
direct PyO3 conversion before expanding Rust coverage to image-heavy paths.
|
|
- Do not expose raw Rust errors that include document contents or upstream
|
|
bodies.
|
|
|
|
## Tests
|
|
|
|
- `cargo test --workspace` must compile this crate.
|
|
- Python tests must cover bridge disabled, bridge enabled, and module-missing
|
|
fallback behavior for every exposed route.
|