litellm/litellm-rust/crates/callbacks-legacy-python/AGENTS.md
devin-ai-integration[bot] e4190d86a6
refactor(rust): centralize host execution and compose callbacks (#43515)
* refactor(rust): extract litellm-host-native as the shared Rust host driver

Move service and hook dispatch out of host-http into a Driver that owns the
machine and Rust handlers, returning at completion or a stream boundary and
holding the demand reply until the consumer advances. Move the in-process
runner onto the same driver. host-http now layers encoding, SSE, body polling
and lifecycle observation over it. host-python keeps driving litellm-host
directly

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust): interrupt the machine when the in-process stream consumer fails

Restores the pre-refactor interruption path for StreamConsumer errors via
Driver::fail and ports the generic run lifecycle tests into host-native.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(rust): separate the machine contract from coroutine execution

* auth update

* refactor(rust): use standard flow control for host requests

* style(rust): keep host driver imports formatted

* chores

* mostly relocation

* refactor(rust): separate interceptors from queued observers

* refactor(rust): centralize legacy callback mappings and lifecycle

* docs: define Python host boundaries and migration plan

* refactor: enforce Python host and bridge boundaries

* refactor(rust): separate operations from callback composition

* refactor(rust): compose SDK policy through call hooks

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-28 19:20:27 +00:00

3.2 KiB

  • Target invariants, not completion claims
  • This crate owns compatibility for all existing Python callbacks and loggers, including CustomLogger. mapping.rs owns the executable call bindings and the inventory of Python-owned hooks. A Python-owned entry records an existing path, never permission to invoke it a second time. The native call adapter preserves the Logging contract (function_setup, the deployment hooks, pre_call/post_call, the sync and async success and failure fan-out, the deferred proxy release, the argument sharing those callbacks rely on)
    • Smell test: if a future callback host (callbacks-v1-python, WASM, in-process Rust) could share a piece of this crate, it does not belong here
    • SDK request policy (credential inheritance, the budget and retry-count limits) is a separate hook supplied by python-bridge; compose it after this adapter so logging adopts the final keyword view before policy mutates or rejects it
    • The driver in litellm-host-python, the routes and core see one PythonCallHooks using the shared CallEvent; they never learn which Python objects consume a call
  • Every litellm Python internal Rust still borrows is a variant of LegacyPython, grouped by subsystem, with its signature pinned in python_contract.json
    • The enum only shrinks: when Rust owns a subsystem, delete its group rather than adding a Rust path beside it
    • Calling a user's own callback directly is permanent Python surface and gets its own type outside LegacyPython
  • PublicCall is the caller's call as Logging sees it: the positional arguments, the keyword view as the call rewrites it (setup, deployment hook, preflight) and the bound request object backing omitted keywords; shared bridge composition hands it to LegacyLogging; routes use the neutral call boundary
  • setup reuses a Logging passed as litellm_logging_obj (the proxy and Router) and otherwise builds one through function_setup; which callbacks run is Logging's decision, never this crate's
  • Callbacks receive the caller's own objects and may mutate them; this crate alone carries that obligation
    • Retain complete boundary arguments, opaque values, aliases, omitted/default distinctions and deliberate copies; preserve the deployment-hook kwargs view
    • Before pre_call, re-alias every body key whose value equals the caller's argument to the caller's own object, resolved through litellm_host_python::lookup
    • Retain body/header roots from pre_call to post_call; in-place mutation reaches the wire, envelope field replacement is visible to later callbacks only
  • Success and failure handlers receive the exact selected public response or exception
    • A failure-handler error cannot suppress the other eligible family or replace the mapped provider error; a cancellation ends the call with no further dispatch
    • Dispatch errors never replay provider work or trigger the opposite outcome; the proxy releases deferred success at most once
    • Delivery follows the registry, not the callable's type: direct, awaited, executor-submitted, logging-worker and deferred paths stay distinct
  • Traverse every retained Python edge; close is idempotent and restores the correlation context once