mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-13 03:59:12 +00:00
ee295aea68
* refactor(rust): give Messages configs typed litellm params * feat(rust): port _normalize_system_role_messages for Messages hosts mid_conversation_system mirrors Python's module function for function: a leading run of system turns is hoisted into the top-level system, a later turn stays in place when the model map flags supports_mid_conversation_system and otherwise becomes a user turn where it was, never between a tool_use and its tool_result, and billing blocks are stripped from the hoisted system. The capability joins MessagesModelCapabilities and the bridge projection. Azure AI uses it in place of fold_system_role_messages, which hoisted every turn. * fix(rust): keep the Bedrock runtime endpoint behind a blank api_base * refactor(rust): import LitellmParams from its crate instead of a re-export * refactor(rust): template the Messages request decode errors Both decode failures go through ErrorDetail::invalid with the subject and the serde error as source instead of a preformatted sentence. * refactor(rust_bridge): read the module globals the param specs name The Messages host no longer keeps its own list of litellm globals; a spec whose spellings the call leaves out names the global to read, so the next family that has one needs no host change. * refactor(rust): give each route lookup Python's implicit None ProviderConfigManager's per-route methods list only the providers the route serves and fall through to None; the Rust lookups now do the same, so a new LlmProviders variant touches only the route that serves it. |
||
|---|---|---|
| .. | ||
| src | ||
| tests | ||
| Cargo.toml | ||
| README.md | ||
litellm-router scaffolds the model-list setup and deployment lookup portion of Python's litellm.Router. Router::from_model_list(&config.model_list) maps configured public names to provider deployments. Programmatic callers can collect (String, Deployment) entries into a Router
Lookup is exact and returns None for an unknown name. This extraction preserves the gateway's existing behavior: the last entry wins when public names repeat. Multiple deployments per model group, routing strategies, retries, cooldowns, and fallbacks are not implemented yet
The router owns deployment configuration and selection. The gateway handles HTTP errors and responses, while the inference-<fmt> crates execute provider calls and resolve credentials