mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-30 01:52:18 +00:00
* feat(rust): expand gateway configuration parsing * refactor(rust): unify core calls and host lifecycle * fix(config): accept environment references for model rate limits Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(rust): unify core calls and host lifecycle * style(rust): apply rustfmt Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(rust): read environment secrets when litellm is not importable Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(core): drop the duplicate rstest attribute Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(rust): make shared route dispatch route-owned * fix(rust): satisfy Clippy in messages regression test --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2.8 KiB
2.8 KiB
- Target invariants, not completion claims
- This crate is the legacy
@clientwrapper as the native call sees it, and nothing else: theLoggingcontract (function_setup, the deployment hooks,pre_call/post_call, the sync and async success and failure fan-out, the deferred proxy release, the argument sharing those callbacks rely on)- Smell test: if a future callback host (
callbacks-v1-python, WASM, in-process Rust) could share a piece of this crate, it does not belong here - SDK request policy (credential inheritance, the budget and retry-count limits) is the driver's preflight, supplied by
python-bridge; this crate only adopts the keyword view it produces - The driver in
litellm-host-python, the routes and core see onePythonCallHooks; they never learn which Python objects consume a call
- Smell test: if a future callback host (
- Every litellm Python internal Rust still borrows is a variant of
LegacyPython, grouped by subsystem, with its signature pinned inpython_contract.json- The enum only shrinks: when Rust owns a subsystem, delete its group rather than adding a Rust path beside it
- Calling a user's own callback directly is permanent Python surface and gets its own type outside
LegacyPython
PublicCallis the caller's call asLoggingsees it: the positional arguments, the keyword view as the call rewrites it (setup, deployment hook, preflight) and the bound request object backing omitted keywords; routes hand it over throughrun_legacy_calland keep no copysetupreuses aLoggingpassed aslitellm_logging_obj(the proxy and Router) and otherwise builds one throughfunction_setup; which callbacks run isLogging's decision, never this crate's- Callbacks receive the caller's own objects and may mutate them; this crate alone carries that obligation
- Retain complete boundary arguments, opaque values, aliases, omitted/default distinctions and deliberate copies; preserve the deployment-hook kwargs view
- Before
pre_call, re-alias every body key whose value equals the caller's argument to the caller's own object, resolved throughlitellm_host_python::lookup - Retain body/header roots from
pre_calltopost_call; in-place mutation reaches the wire, envelope field replacement is visible to later callbacks only
- Success and failure handlers receive the exact selected public response or exception
- A failure-handler error cannot suppress the other eligible family or replace the mapped provider error; a cancellation ends the call with no further dispatch
- Dispatch errors never replay provider work or trigger the opposite outcome; the proxy releases deferred success at most once
- Delivery follows the registry, not the callable's type: direct, awaited, executor-submitted, logging-worker and deferred paths stay distinct
- Traverse every retained Python edge;
closeis idempotent and restores the correlation context once