litellm/litellm-rust/crates/python-bridge/src/cache/python/mod.rs
Yujong Lee d9e069d7bb
Some checks failed
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
refactor(rust-bridge): serve the Python cache around native calls
Python owns the response cache for native calls: the host driver looks the call up after argument preparation and before the native machine starts, delivers a hit without touching the provider, and stores the public response the deployment success hook answered or the stream the caller read to its end. Hits and hook-reviewed responses no longer pass through native response types, so fields Rust does not model survive. The ProviderResult intercept and the Python-backed Rust cache service are removed.

Co-authored-by: Yujong Lee <yujong@berri.ai>
2026-10-24 15:28:46 -07:00

3 lines
69 B
Rust

mod binding;
pub(crate) use binding::{CachedBinding, CachedStream};