scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-30 01:52:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 70
litellm/litellm-rust/crates/host-python/AGENTS.md
devin-ai-integration[bot] 36784e3b79
refactor(rust): share call lifecycle across route-owned inference (#43461) 
* feat(rust): expand gateway configuration parsing

* refactor(rust): unify core calls and host lifecycle

* fix(config): accept environment references for model rate limits

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(rust): unify core calls and host lifecycle

* style(rust): apply rustfmt

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust): read environment secrets when litellm is not importable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(core): drop the duplicate rstest attribute

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(rust): make shared route dispatch route-owned

* fix(rust): satisfy Clippy in messages regression test

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-27 16:05:30 -07:00

3.1 KiB
Raw Blame History

  • Target invariants; implementation and runtime validation may lag these rules
  • Keep this crate the CPython runtime adapter and nothing more: Serde marshalling, interpreter detachment, tokio/asyncio glue, the Execution handle, the call driver and the PythonBinding, PythonHostCalls, PythonCallHooks and PythonOwned traits
    • No LiteLLM domain dependencies beyond litellm-host: no route types, no Logging policy, no public API registration, no cdylib build features
    • The driver emits Succeeded or Failed exactly once and never dispatches after a cancellation; which Python objects consume those events is the lifecycle's business
    • PythonBinding::decode_request receives the keyword view the hooks' prepare_arguments returned, rewritten in place by the route's Preflight, not the caller's dict; a binding that decodes from it inherits the lifecycle's rewrites (for the legacy adapter: setup, deployment hooks) and the preflight's (credential inheritance)
    • A native failure, including one a host op returns as InvokeError::Native, is classified exactly once through the binding's map_error; a Python exception raised inside the call, and a failure in prepare_arguments or transform_response, is raised as is
    • A failing map_error is raised with the native error's text as its __context__, never swallowed
  • Use standard PyO3 ownership and conversion APIs
    • Prefer Bound<'py, T> for attached operations/results, Py<T> for retention; binding/unbinding does not copy payloads
    • Use pythonize for selected Serde data, never a JSON-text round trip; share conversion with Pythonized<T>
    • Preserve PythonizeError's standard conversion into PyErr; do not stringify original Python exceptions into new ValueErrors
    • Keep serializer-panic containment in Pythonized<T>: async output conversion can run in an unjoined blocking task and otherwise strand delivery
  • Use Python::detach for Rust-only work; Python operations require attachment
    • Keep diagnostic counters in the consumer; wrapper invocations do not measure every interpreter release
    • Release exclusive class borrows/locks before Python calls or decrements that can invoke finalizers; expose retained Python edges to GC without calling Python during traversal
  • Keep coroutine driving in the shared Python driver and the native handle
    • Driver: litellm/rust_bridge/lifecycle.py; handle: src/handle.rs; call driver: src/driver.rs; native-backed behavior tests: tests/lifecycle.py
    • Every lifecycle suspension is awaited inline in the caller's task; into_future creates a separate task and cannot satisfy this contract
  • References: ownership, conversions, pythonize errors