litellm/litellm-rust/crates/host-python/AGENTS.md

• Target invariants. Implementation and runtime validation may lag these rules

Boundary with Python consumers

This crate owns CPython execution mechanics for generic litellm-host machines and hooks. Consumers supply domain bindings, host operations, public result construction and callback policy. Neither Rust dependencies nor Python imports may require LiteLLM route modules or legacy Logging

src/native.rs belongs here: it runs a generic machine through the Python runtime and owns its pending execution and abort handle. Keep provider selection, request projection and public exception policy out of it. A rename to machine_runner.rs is optional and must not change behavior

run_call returns the unstarted Execution and never names a Python stream class or driver. The consumer drives it from Python and supplies the step classes through StepTypes rather than have this crate import a fixed litellm.rust_bridge module. Generic suspension and execution state validation belong here. Public stream wrappers and _hidden_params conventions belong to the consumer

Creating a resolved asyncio Future from an already constructed Python value belongs here, alongside runtime waiting, interpreter detachment and panic containment. Choosing which callable exceptions become a public RuntimeError belongs to the consumer. python-bridge::callable::wrap_failure owns that policy

The driver owns ordering: start, argument preparation, prepared-argument hooks, binding decode and machine start. Fallible per-call resource setup supplied by the consumer runs after all argument hooks, using the prepared argument view, and before provider work. Setup failure follows the existing terminal failure path. Creating or discarding an unstarted coroutine must not initialize clients, acquire credentials or capture execution context

Boundary tests exercise behavior with a supplied lifecycle binding without importing the LiteLLM Python package. Pin inline awaiting, awaitable final values, exception identity, cancellation, re-entry and release of retained objects, rather than module names or source layout

PythonSurface is the shared Surface for Python: Py<PyDict> arguments, Py<PyAny> responses and chunks, PyErr errors, the caller's task as &dyn Awaiter. A group implements the four facets (ArgumentHooks, ProviderHooks, ResponseHooks, OutcomeHooks) plus PythonOwned and is mounted with PythonSurface::group, which guards every stage: a cancellation the task resumes a hook with ends that stage there. Hooks is the shared chain on this surface, and the bridge composes it; SDK policy is a mounted group, never a driver phase. A group implements only the stages it needs, default stages preserve the supplied values. The driver calls prepared on every group after all prepare stages, with the final keyword view

Terminal outcomes share the selected response or exception. A Python outcome hook settles its own result through report_unless_cancelled: an ordinary exception is written unraisable and does not skip the next group or replace the selected outcome, anything else is Cancelled and stops all further dispatch. Preparation, interception and transformation errors stop the chain. Suspensions stay inline in the existing driver

Existing runtime invariants

• Keep this crate the CPython runtime adapter and nothing more: Serde marshalling, interpreter detachment, tokio/asyncio glue, the Execution handle, the call driver and the PythonBinding, PythonHostCalls and PythonOwned traits, and the PythonSurface specialization of the host::hooks facets
  • No LiteLLM domain dependencies beyond litellm-host: no route types, no Logging policy, no public API registration, no cdylib build features
  • PythonGroup only constrains the shared facets to PythonSurface and Python ownership. It must not redeclare the stages
  • The driver delivers succeeded or failed exactly once and never dispatches callbacks after a cancellation. Which Python objects consume those outcomes is the legacy adapter's business
  • CallOptions can publish snapshots independently of callback delivery. Terminal snapshots follow completed hook dispatch, and cancellation never calls a Python callback. For Python-driven calls, leave the machine's observation publisher unset so the driver is the sole publisher of intercepted provider-response snapshots
  • PythonBinding::decode_request receives the keyword view returned by prepare and seen by prepared, not the caller's dict. A binding that decodes from it inherits all composed argument rewrites
  • A native failure, including one a host op returns as InvokeError::Native, is classified exactly once through the binding's map_error. A Python exception raised inside the call, and a failure in prepare or transform, is raised as is
  • A failing map_error is raised with the native error's text as its __context__, never swallowed
• Use standard PyO3 ownership and conversion APIs
  • Prefer Bound<'py, T> for attached operations/results, Py<T> for retention. Binding/unbinding does not copy payloads
  • Use pythonize for selected Serde data, never a JSON-text round trip. Share conversion with Pythonized<T>
  • Preserve PythonizeError's standard conversion into PyErr. Do not stringify original Python exceptions into new ValueErrors
  • Keep serializer-panic containment in Pythonized<T>: async output conversion can run in an unjoined blocking task and otherwise strand delivery
• Use Python::detach for Rust-only work. Python operations require attachment
  • Keep diagnostic counters in the consumer. Wrapper invocations do not measure every interpreter release
  • Release exclusive class borrows/locks before Python calls or decrements that can invoke finalizers. Expose retained Python edges to GC without calling Python during traversal
• Keep coroutine driving in the shared Python driver and the native handle
  • Shared driver implementation: litellm/rust_bridge/execution.py, handle: src/handle.rs, call driver: src/driver.rs, native-backed behavior tests: tests/lifecycle.py. The consumer supplies the lifecycle binding
  • Every lifecycle suspension is awaited inline in the caller's task. into_future creates a separate task and cannot satisfy this contract
• References: ownership, conversions, pythonize errors
  • GC, re-entry, parallelism, async delivery source