Non-streaming responses served by the Rust core already carry
x-litellm-rust: true through _hidden_params.additional_headers, which the
SDK exposes and the gateway renders as a response header. Native streams
did not, because the lifecycle Stream and SyncStream objects had nowhere
to hold hidden params and the marker writer skips objects without them.
Give both stream classes the same _hidden_params bag every other litellm
response has, so the existing marker attaches without wrapping the stream
or changing its identity.
Co-authored-by: Yujong Lee <yujong@berri.ai>
* docs(rust): plan Python interop foundation
* fix(rust): preserve Python settings coercion at the native boundary
* chore(rust): drop interop planning note
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): resolve OCR provider secrets through an async SecretSource before transformation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): project the Python secret manager into the bridge and resolve OCR secrets through it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust): drop premium_user from the secret manager snapshot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust-bridge): read the private key management globals once in the settings snapshot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(rust): bound the bridge secret manager state cache to the active snapshot
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(rust): inline coercion unit tests
* fix(rust): preserve Python secret manager bindings
* refactor(rust-bridge): let settings projectors own their contract specs
Each settings group now declares its SettingSpec rows next to the projector
that reads them, and the manifest test derives python_settings.json from those
tables instead of a hand-copied duplicate. Field carries (group, name) instead
of a dotted path, and coercion gains the dict-item reader plus the Redis
Boolean, certificate-requirement, non-empty string, and numeric adapters that
the cache configuration projection adopts next.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust-bridge): capture the secret manager binding in one settings read
The secret_manager accessor now carries the live client and settings objects,
so the bridge classifies the binding from a single snapshot instead of
re-reading litellm globals. The unreachable native arm and the service alias
go away, the binding-to-state mapping moves next to the snapshot, and the
Python callback precomputes its key_manager name.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust-bridge): execute typed settings field declarations
* refactor(rust-bridge): compare cache backends by identity behind one exact trait
cache-response gains an object-safe ExactResponseCache so every exact-match
backend sits behind one pointer; WriteBuffer flushes through it. The bridge's
NativeResponseCache shrinks from nine variants and fifteen per-backend
accessors to an exact service plus the three semantic backends, and facade
mismatch detection compares BackendIdentity values instead of matching on
each backend type. Request projections move next to NativeRequest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(rust-bridge): drive both Python-embedded semantic caches through one execution
Redis-semantic and Valkey-semantic operations now share one SemanticExecution
body: await the Python embedder, seed the task-local vector, run the native
backend, repeat per batch entry. Valkey drops its with_embedder path in favor
of the same seeded embedder, and each backend keeps its own embedding-failure
policy. PythonEmbedder exposes one call shape. Redis-semantic thresholds are
compared at the backend's f32 width, which un-breaks the redis-stack parity
tests that a 0.8 facade threshold failed before this branch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* wip
* feat(rust-bridge): complete response cache runtime surface
* fix(rust-bridge): preserve secret manager callback exceptions
* refactor(rust-bridge): unify route cache and secret rollout catalog
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Parse the switch with pydantic TypeAdapter(bool) so 1/true/yes/on and 0/false/no/off all work, and treat an unparseable value as unset instead of off. PYTHON_ONLY and RUST_REQUIRED still ignore both switches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replace the per-route enablement helpers (rust_enabled, rust_ocr_enabled, RUST_CHAT_COMPLETIONS_PROVIDERS, FallbackMode) with a single rule table in litellm/rust_bridge/catalog.py that maps a Context(route, provider, model, delivery) to one of four rollout tiers, and a pure decide() that turns tier plus process/env switches into a Decision. runtime.run/arun own the only fallback path: Python for PYTHON, native then Python on missing binding or admission decline for RUST_WITH_FALLBACK, raise for RUST_REQUIRED. OCR is the first route on the shared runtime; chat completions, Anthropic messages, and Responses websocket policy checks now read the catalog.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
