scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-12 23:01:41 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 87
litellm/litellm-rust/ADDING_A_PROVIDER.md
ishaan-berri 18406c2bad
feat: add openai realtime translation layer to litellm-rust (1/2) (#31129) 
* add RealtimeTransformResult type for realtime transforms

* add RealtimeProviderConfig pure trait in litellm-core

* add core realtime module

* register realtime module in litellm-core lib

* add OpenAI realtime transform + complete_url parity in providers

* add openai realtime module

* add openai provider module

* register openai provider module in providers lib

* add realtime() fn that invokes OpenAI GA realtime API end to end

* register realtime route module in providers lib

* wire tokio/tokio-tungstenite/futures-util into providers crate

* add tokio, tokio-tungstenite, futures-util to rust workspace deps

* update Cargo.lock for realtime websocket deps

* docs: add litellm-rust provider/route contributor guide

* add typed RealtimeEvent; make RealtimeTransformResult hold typed events

* type RealtimeProviderConfig trait on RealtimeEvent instead of raw strings

* type OpenAI realtime passthrough transforms on RealtimeEvent

* type realtime() fn on RealtimeEvent end to end (parse/serialize at host edge)

* docs: add typed-contracts core rule to core CLAUDE.md

* harden complete_url: default bare host / unknown scheme to wss://

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
2026-06-23 15:48:51 -07:00

1.1 KiB
Raw Permalink Blame History

Adding a provider / route to litellm-rust

Three layers, same for every route (see ocr and realtime as references):

  1. Transform contract (pure) — crates/core/src/<route>/transformation.rs: a …ProviderConfig trait (URL build + request/response transforms) + types in types.rs. No network, env, or auth.
  2. Provider config (pure) — crates/providers/src/<provider>/<route>/transformation.rs: implement that trait as a const <PROVIDER>_<ROUTE>_CONFIG, mirroring the Python provider tree. Add parity unit tests.
  3. HTTP / transport (the host) — crates/providers/src/<route>.rs (e.g. ocr.rs, realtime.rs): the callable fn (run_ocr, realtime). It resolves the key, builds the auth header, builds URL + transforms via the config, then does the network call. This is the only layer allowed to do I/O.

Calling: the host invokes the route fn — the Python bridge calls run_ocr; the ai-gateway server calls realtime. Register new modules in lib.rs / mod.rs, then run cargo fmt && cargo clippy --workspace -- -D warnings && cargo test --workspace.