scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-27 01:22:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 80
litellm/litellm-rust/crates/host-python/AGENTS.md
devin-ai-integration[bot] e9491d31b5
Some checks are pending
CI Coverage / assert-ci-coverage (push) Waiting to run
Details
CodeQL / Analyze (actions) (push) Waiting to run
Details
CodeQL / Analyze (javascript-typescript) (push) Waiting to run
Details
CodeQL / Analyze (python) (push) Waiting to run
Details
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Details
Helm unit test / unit-test (push) Waiting to run
Details
Publish basedpyright base counts / publish (push) Waiting to run
Details
Scorecard supply-chain security / Scorecard analysis (push) Waiting to run
Details
Code Quality Checks / code-quality (push) Waiting to run
Details
Code Quality Checks / python-310-import-smoke (push) Waiting to run
Details
UI Unit Tests / ui-unit-tests (push) Waiting to run
Details
Postgres Tests / proxy-security (push) Waiting to run
Details
Postgres Tests / schema-migration (push) Waiting to run
Details
Postgres Tests / proxy-behavior (push) Waiting to run
Details
LiteLLM Rust / rust-lint (push) Waiting to run
Details
LiteLLM Rust / rust-test (push) Waiting to run
Details
LiteLLM Rust / rust-wheel (push) Waiting to run
Details
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Details
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Details
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Details
Unit Tests / proxy-endpoints (push) Waiting to run
Details
Unit Tests / caching-local (push) Waiting to run
Details
Unit Tests / core-utils (push) Waiting to run
Details
Unit Tests / enterprise-package (push) Waiting to run
Details
Unit Tests / enterprise-routing (push) Waiting to run
Details
Unit Tests / integrations (push) Waiting to run
Details
Unit Tests / All Other Providers (push) Waiting to run
Details
Unit Tests / Vertex AI (push) Waiting to run
Details
Unit Tests / mcp-integration (push) Waiting to run
Details
Unit Tests / misc (push) Waiting to run
Details
Unit Tests / proxy-auth (push) Waiting to run
Details
Unit Tests / proxy-extras (push) Waiting to run
Details
Unit Tests / proxy-infra (push) Waiting to run
Details
Unit Tests / proxy-server (push) Waiting to run
Details
Unit Tests / responses-caching-types (push) Waiting to run
Details
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Details
refactor(rust): move credential inheritance and the SDK limits out of the legacy callback crate into a driver preflight (#43259) 
The legacy callback crate carried two rewrites that have nothing to do with the
Logging contract: litellm_credential_name inheritance and the max_budget and
num_retries_per_request checks. Any later callback host would need them
unchanged, which is the smell the crate's AGENTS.md now names. They are now a
Preflight the driver in litellm-host-python runs on the keyword view begin
returned, before the host projects from it, supplied by python-bridge and passed
through run_legacy_call. The call order is unchanged (setup, deployment hook,
credentials, limits) and a rejection still fails the call as a host failure, so
the failure callbacks run as before. The preflight rewrites the adapter's own
copy in place, so no extra dict copy and no new lifecycle method

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-26 01:09:36 +00:00

3 KiB
Raw Blame History

  • Target invariants; implementation and runtime validation may lag these rules
  • Keep this crate the CPython runtime adapter and nothing more: Serde marshalling, interpreter detachment, tokio/asyncio glue, the Execution handle, the call driver and the PythonLifecycle/ProtocolHost traits
    • No LiteLLM domain dependencies beyond litellm-host: no route types, no Logging policy, no public API registration, no cdylib build features
    • The driver emits Succeeded or Failed exactly once and never dispatches after a cancellation; which Python objects consume those events is the adapter's business
    • ProtocolHost::project receives the keyword view the adapter's begin returned, rewritten in place by the route's Preflight, not the caller's dict; a protocol host that projects from it inherits the adapter's rewrites (for the legacy adapter: setup, deployment hooks) and the preflight's (credential inheritance)
    • A native failure, including one a host op returns as InvokeError::Native, is classified exactly once through the route's classify; a Python exception raised inside the call, and a failure in begin or after_success, is raised as is
    • A failing classify is raised with the native error's text as its __context__, never swallowed
  • Use standard PyO3 ownership and conversion APIs
    • Prefer Bound<'py, T> for attached operations/results, Py<T> for retention; binding/unbinding does not copy payloads
    • Use pythonize for selected Serde data, never a JSON-text round trip; share conversion with Pythonized<T>
    • Preserve PythonizeError's standard conversion into PyErr; do not stringify original Python exceptions into new ValueErrors
    • Keep serializer-panic containment in Pythonized<T>: async output conversion can run in an unjoined blocking task and otherwise strand delivery
  • Use Python::detach for Rust-only work; Python operations require attachment
    • Keep diagnostic counters in the consumer; wrapper invocations do not measure every interpreter release
    • Release exclusive class borrows/locks before Python calls or decrements that can invoke finalizers; expose retained Python edges to GC without calling Python during traversal
  • Keep coroutine driving in the shared Python driver and the native handle
    • Driver: litellm/rust_bridge/lifecycle.py; handle: src/handle.rs; call driver: src/driver.rs; native-backed behavior tests: tests/lifecycle.py
    • Every adapter suspension is awaited inline in the caller's task; into_future creates a separate task and cannot satisfy this contract
  • References: ownership, conversions, pythonize errors