litellm/litellm-rust/crates/callbacks-legacy-python/AGENTS.md
Yujong Lee d187fe8b2c refactor(rust): compose callback lifecycles in a chain and tighten both contracts
Compose the legacy and v1 lifecycles behind one PythonLifecycle through
LifecycleChain, built once in python-bridge, and simplify the boundary:

- drop LifecycleEvent::Started; begin carries the start time and is the first
  thing a lifecycle hears of a call
- give PythonLifecycle pass-through defaults for the threading steps and the
  stream notices, and document which steps thread and which observe
- export one is_cancellation from litellm-host-python
- callbacks-v1: private RequestFacts fields, a golden file per EventKind by
  construction, Python EVENTS and SUPPORTED_SCHEMAS pinned to the goldens
- callbacks-v1-python: the snapshot only parses (register is the only place
  that rejects), Handler owns call/is_async, Pending keeps a subscriber index
- callbacks-legacy-python: LegacyCallbacks becomes inherent PythonLogger
  methods, deployment hooks become free functions, retained headers are
  traversed and cleared
- define the shared terms in host-python/AGENTS.md and each contract's own
  terms in its crate
- remove test_loader.py, whose fork hook was never committed

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-19 15:12:53 -07:00

4.6 KiB

  • Target invariants, not completion claims
  • Keep this crate the legacy @client wrapper as the native call sees it, and nothing else: the Logging contract (function_setup, the deployment hooks, pre_call/post_call, the sync and async success and failure fan-out, the deferred proxy release, the argument sharing those callbacks rely on) plus the kwargs rewrites the wrapper makes on the way in (credential-name inheritance, the budget and retry-count limits)
    • The driver in litellm-host-python, the routes and core see one PythonLifecycle; they never learn which Python objects consume a call
  • Terms (lifecycle terms are in host-python/AGENTS.md)
    • Logging: litellm's Python class; logger: the one Logging instance of this call (PythonLogger), reused from litellm_logging_obj or built by function_setup
    • Borrowed function: a litellm Python internal Rust still calls, one variant of LegacyPython; a user's own callback is never one
    • Fan-out: Logging dispatching to every registered callback; family: the sync or the async set of success and failure handlers
    • Deployment hooks: the awaited pre-call, post-call and failure hooks on each callback; async calls only, as in @client
    • Re-aliasing: replacing body values with the caller's equal objects before pre_call, so a callback mutates what the caller holds
    • Deferred release: the proxy holds back async success logging until it accepts the response (PendingLogging.release)
    • Internal call: a proxy-internal call; when it is async, the async success dispatch and both failure families are skipped, and the sync success handlers still run
    • Pass-through billing (PassThroughStream): end-of-stream cost logging from the delivered chunks; a failure after the stream opened bills them as partial usage, and a sync call falls back to the plain failure handler
    • Redaction here is ****, matching what the Python path logs; [REDACTED] belongs to the v1 contract
  • Rust drives the call; every litellm Python internal it still borrows is a variant of LegacyPython, grouped by subsystem (Wrapper, Logging, DeploymentHooks, Streaming)
    • The enum only shrinks: when Rust owns a subsystem, delete its group rather than adding a Rust path beside it
    • Calling a user's own callback directly is permanent Python surface and gets its own type outside LegacyPython
    • PublicCall is the caller's call as Logging sees it: the positional arguments, the keyword view as the legacy path rewrites it (setup, deployment hook, prepare) and the bound request object whose attributes back keywords the caller omitted; routes capture it and the bridge's run_python_call hands it to LegacyPythonLifecycle::new, keeping no copy
  • setup reuses a Logging the caller passed as litellm_logging_obj (the proxy and Router are the live cases) and otherwise builds one through function_setup, as @client does
    • Either way every phase calls the same Logging method the Python path calls; which callbacks run is Logging's decision, never this crate's
  • Callbacks receive the caller's own objects and may mutate them; this crate alone carries that obligation
    • Retain complete boundary arguments, opaque unknown values, aliases, omitted/default distinctions and deliberate copies; preserve the established deployment-hook kwargs view
    • Before pre_call, re-alias every body key whose value equals the caller's argument to the caller's own object; this crate compares the two itself, and the argument is resolved by litellm_host_python::lookup
    • Retain independently captured body/header roots from pre_call to post_call; in-place mutation reaches the wire, envelope field replacement is visible to later callbacks only
    • A later kind of callback host (WASM, in-process Rust) has none of these obligations, so they stay out of litellm-host, litellm-host-python and the bridge; the only fact that crosses from the route is the prepared keyword view
  • Success and failure handlers receive the exact selected public response or exception; logging projections, redaction and snapshots keep their own copy contracts
    • Ordinary failure-handler errors cannot suppress the other eligible family or replace the mapped provider error; a cancellation ends the call with no further dispatch
    • Dispatch errors never replay provider work or trigger the opposite outcome; the proxy's acceptance or rejection releases deferred success at most once
    • Delivery follows the registry, not the callable's type: direct, awaited, executor-submitted, logging-worker and deferred paths stay distinct
  • Traverse every retained Python edge; close is idempotent and restores the correlation context once