mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-05 02:41:56 +00:00
18406c2bad
* add RealtimeTransformResult type for realtime transforms * add RealtimeProviderConfig pure trait in litellm-core * add core realtime module * register realtime module in litellm-core lib * add OpenAI realtime transform + complete_url parity in providers * add openai realtime module * add openai provider module * register openai provider module in providers lib * add realtime() fn that invokes OpenAI GA realtime API end to end * register realtime route module in providers lib * wire tokio/tokio-tungstenite/futures-util into providers crate * add tokio, tokio-tungstenite, futures-util to rust workspace deps * update Cargo.lock for realtime websocket deps * docs: add litellm-rust provider/route contributor guide * add typed RealtimeEvent; make RealtimeTransformResult hold typed events * type RealtimeProviderConfig trait on RealtimeEvent instead of raw strings * type OpenAI realtime passthrough transforms on RealtimeEvent * type realtime() fn on RealtimeEvent end to end (parse/serialize at host edge) * docs: add typed-contracts core rule to core CLAUDE.md * harden complete_url: default bare host / unknown scheme to wss:// --------- Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
9 lines
1.1 KiB
Markdown
9 lines
1.1 KiB
Markdown
# Adding a provider / route to litellm-rust
|
|
|
|
Three layers, same for every route (see `ocr` and `realtime` as references):
|
|
|
|
1. **Transform contract (pure)** — `crates/core/src/<route>/transformation.rs`: a `…ProviderConfig` trait (URL build + request/response transforms) + types in `types.rs`. No network, env, or auth.
|
|
2. **Provider config (pure)** — `crates/providers/src/<provider>/<route>/transformation.rs`: implement that trait as a `const <PROVIDER>_<ROUTE>_CONFIG`, mirroring the Python provider tree. Add parity unit tests.
|
|
3. **HTTP / transport (the host)** — `crates/providers/src/<route>.rs` (e.g. `ocr.rs`, `realtime.rs`): the callable fn (`run_ocr`, `realtime`). It resolves the key, builds the auth header, builds URL + transforms via the config, then does the network call. This is the only layer allowed to do I/O.
|
|
|
|
**Calling:** the host invokes the route fn — the Python bridge calls `run_ocr`; the `ai-gateway` server calls `realtime`. Register new modules in `lib.rs` / `mod.rs`, then run `cargo fmt && cargo clippy --workspace -- -D warnings && cargo test --workspace`.
|