Commit graph

231 commits

Author SHA1 Message Date
devin-ai-integration[bot]
3106d9c573
feat(rust-bridge): add cache and secret migration foundations (#42328)
* docs(rust): plan Python interop foundation

* fix(rust): preserve Python settings coercion at the native boundary

* chore(rust): drop interop planning note

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(rust): resolve OCR provider secrets through an async SecretSource before transformation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(rust): project the Python secret manager into the bridge and resolve OCR secrets through it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(rust): drop premium_user from the secret manager snapshot

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust-bridge): read the private key management globals once in the settings snapshot

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(rust): bound the bridge secret manager state cache to the active snapshot

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(rust): inline coercion unit tests

* fix(rust): preserve Python secret manager bindings

* refactor(rust-bridge): let settings projectors own their contract specs

Each settings group now declares its SettingSpec rows next to the projector
that reads them, and the manifest test derives python_settings.json from those
tables instead of a hand-copied duplicate. Field carries (group, name) instead
of a dotted path, and coercion gains the dict-item reader plus the Redis
Boolean, certificate-requirement, non-empty string, and numeric adapters that
the cache configuration projection adopts next.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(rust-bridge): capture the secret manager binding in one settings read

The secret_manager accessor now carries the live client and settings objects,
so the bridge classifies the binding from a single snapshot instead of
re-reading litellm globals. The unreachable native arm and the service alias
go away, the binding-to-state mapping moves next to the snapshot, and the
Python callback precomputes its key_manager name.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(rust-bridge): execute typed settings field declarations

* refactor(rust-bridge): compare cache backends by identity behind one exact trait

cache-response gains an object-safe ExactResponseCache so every exact-match
backend sits behind one pointer; WriteBuffer flushes through it. The bridge's
NativeResponseCache shrinks from nine variants and fifteen per-backend
accessors to an exact service plus the three semantic backends, and facade
mismatch detection compares BackendIdentity values instead of matching on
each backend type. Request projections move next to NativeRequest.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(rust-bridge): drive both Python-embedded semantic caches through one execution

Redis-semantic and Valkey-semantic operations now share one SemanticExecution
body: await the Python embedder, seed the task-local vector, run the native
backend, repeat per batch entry. Valkey drops its with_embedder path in favor
of the same seeded embedder, and each backend keeps its own embedding-failure
policy. PythonEmbedder exposes one call shape. Redis-semantic thresholds are
compared at the backend's f32 width, which un-breaks the redis-stack parity
tests that a 0.8 facade threshold failed before this branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* wip

* feat(rust-bridge): complete response cache runtime surface

* fix(rust-bridge): preserve secret manager callback exceptions

* refactor(rust-bridge): unify route cache and secret rollout catalog

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-22 03:41:04 +00:00
Yujong Lee
e3ca80b19a Merge origin/main into litellm_rust_qdrant_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 00:19:34 +00:00
Yujong Lee
c31fa2ae7a Merge remote-tracking branch 'origin/main' into litellm_native_redis_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:51:16 +00:00
Yujong Lee
2eee880aba Merge origin/main into litellm_valkey_semantic_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:18:54 +00:00
yujonglee
a5431244ef
Merge pull request #42313 from BerriAI/litellm_rust_cache_s3
feat(rust): add native S3 cache backend
2026-09-21 16:12:54 -07:00
Yujong Lee
3122d21ea6 Merge origin/main into litellm_rust_qdrant_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:09:55 +00:00
Yujong Lee
615a226a30 Merge remote-tracking branch 'origin/main' into litellm_native_redis_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:08:37 +00:00
Yujong Lee
4c9e45f5f1 Merge origin/main into litellm_valkey_semantic_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:08:31 +00:00
Yujong Lee
0c5bbf8c37 Merge branch 'main' into litellm_rust_cache_s3
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 23:04:42 +00:00
yujonglee
ced2980529
Merge pull request #42308 from BerriAI/litellm_rust_secrets_hashicorp_vault
feat(rust): add HashiCorp Vault secret manager crate
2026-09-21 16:03:33 -07:00
Yujong Lee
eec133dc27 Merge origin/main into litellm_rust_qdrant_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:56:07 +00:00
Yujong Lee
0116a502bc Merge remote-tracking branch 'origin/main' into litellm_native_redis_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:53:48 +00:00
Yujong Lee
90f93f76f4 Merge origin/main into litellm_valkey_semantic_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:53:27 +00:00
Yujong Lee
911e320b95 Merge remote-tracking branch 'origin/main' into litellm_rust_cache_s3
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm-rust/Cargo.lock
#	litellm-rust/Cargo.toml
#	litellm-rust/crates/python-bridge/Cargo.toml
#	litellm-rust/crates/python-bridge/src/cache/config.rs
#	litellm-rust/crates/python-bridge/src/cache/facade.rs
#	litellm-rust/crates/python-bridge/src/cache/handle.rs
#	litellm-rust/crates/python-bridge/src/cache/native.rs
#	tests/test_litellm_rust/test_cache.py
2026-09-21 22:53:07 +00:00
Yujong Lee
636a60d3bb chore(rust): update merged workspace lockfile
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:51:14 +00:00
Yujong Lee
4e2047e9c5 chore: merge main into litellm_gcs_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:51:05 +00:00
Yujong Lee
ae32609b54 refactor(cache-qdrant-semantic): inject the shared LiteLLM HTTP client into the embedder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:18:13 +00:00
Yujong Lee
dd175245cf Merge origin/main into litellm_valkey_semantic_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:13:55 +00:00
Yujong Lee
2129a94e56 Merge remote-tracking branch 'origin/main' into litellm_native_redis_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:13:38 +00:00
Yujong Lee
f7b96c9ced Merge origin/main into litellm_native_disk_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:13:09 +00:00
Yujong Lee
5ca7968812 Merge remote-tracking branch 'origin/main' into litellm_rust_cache_s3
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm-rust/crates/python-bridge/src/cache/config.rs
#	litellm-rust/crates/python-bridge/src/cache/facade.rs
#	litellm-rust/crates/python-bridge/src/cache/handle.rs
#	litellm-rust/crates/python-bridge/src/cache/native.rs
2026-09-21 22:12:40 +00:00
Yujong Lee
bda406c0af Merge remote-tracking branch 'origin/main' into litellm_rust_qdrant_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm-rust/crates/python-bridge/src/cache/config.rs
#	litellm-rust/crates/python-bridge/src/cache/facade.rs
#	litellm-rust/crates/python-bridge/src/cache/handle.rs
#	litellm-rust/crates/python-bridge/src/cache/native.rs
2026-09-21 22:10:05 +00:00
Yujong Lee
cb3e212e29 chore: merge main into litellm_gcs_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 22:08:55 +00:00
yujonglee
844d24b095
Merge pull request #42321 from BerriAI/litellm_native_azure_blob_cache
feat(rust): native Azure Blob response cache backend
2026-09-21 15:06:55 -07:00
yujonglee
41adbdaa05
Merge pull request #42309 from BerriAI/litellm_rust_secrets_azure_key_vault
feat(rust): add Azure Key Vault secret manager backend
2026-09-21 15:04:10 -07:00
Yujong Lee
cf7234c6f4 chore(rust): refresh workspace lockfile
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:48:40 +00:00
Yujong Lee
c3c8c35cb2 Merge origin/main into litellm_valkey_semantic_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:48:32 +00:00
Yujong Lee
93e9836524 feat(python-bridge): serve QdrantSemanticCache natively
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:47:56 +00:00
Yujong Lee
8d41336a1e test(cache-qdrant-semantic): cover backend contract against an in-process Qdrant
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:47:56 +00:00
Yujong Lee
80c0ceb5e6 feat(cache-qdrant-semantic): add native Qdrant semantic cache backend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:47:56 +00:00
Yujong Lee
63c16fe5ac Merge origin/main into litellm_native_disk_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:47:35 +00:00
Yujong Lee
1509afe77e Merge remote-tracking branch 'origin/main' into litellm_gcs_native_cache 2026-09-21 21:47:29 +00:00
Yujong Lee
9f4448f42f chore(rust): merge main into litellm_gcs_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:47:29 +00:00
Yujong Lee
95c1ba5e1d Merge remote-tracking branch 'origin/main' into litellm_native_redis_semantic_cache 2026-09-21 21:47:22 +00:00
Yujong Lee
b926cf998a chore: merge main into litellm_rust_secrets_azure_key_vault
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:47:21 +00:00
Yujong Lee
bc641f01ae Merge remote-tracking branch 'origin/main' into litellm_native_azure_blob_cache 2026-09-21 21:47:19 +00:00
Yujong Lee
f24244a1a4 Merge remote-tracking branch 'origin/main' into litellm_native_redis_semantic_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm-rust/crates/cache-redis/src/cache.rs
#	litellm-rust/crates/cache-redis/src/cache/operations.rs
#	litellm-rust/crates/python-bridge/src/cache/config.rs
#	litellm-rust/crates/python-bridge/src/cache/facade.rs
#	litellm-rust/crates/python-bridge/src/cache/handle.rs
#	litellm-rust/crates/python-bridge/src/cache/native.rs
#	tests/test_litellm_rust/test_cache.py
2026-09-21 21:47:12 +00:00
yujonglee
ebb4d230f8
Merge pull request #42300 from BerriAI/litellm_python_interop_foundation
fix(rust): preserve Python settings semantics at the native boundary
2026-09-21 14:46:37 -07:00
Yujong Lee
3ebbb2af5b Merge remote-tracking branch 'origin/main' into litellm_rust_cache_s3
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

# Conflicts:
#	litellm-rust/Cargo.lock
#	litellm-rust/crates/python-bridge/src/cache/config.rs
#	litellm-rust/crates/python-bridge/src/cache/facade.rs
#	litellm-rust/crates/python-bridge/src/cache/handle.rs
#	tests/test_litellm_rust/test_cache.py
2026-09-21 21:46:22 +00:00
yujonglee
162e5c4338
Merge pull request #42303 from BerriAI/litellm_rust_secrets_cyberark
feat(rust): add CyberArk Conjur secret manager backend
2026-09-21 14:45:39 -07:00
Yujong Lee
2f54f6078d chore: merge main into litellm_native_azure_blob_cache
Combines the Redis cluster bridge changes with the Azure Blob variant and
keeps account URL query parameters when building the container URL

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:43:08 +00:00
Yujong Lee
98e9e8c602 Merge origin/main into litellm_native_disk_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:42:10 +00:00
Yujong Lee
cc6ca5ee04 Merge origin/main into litellm_valkey_semantic_native_cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:32:33 +00:00
Yujong Lee
52216df1ff fix(rust): preserve structured conversion semantics 2026-09-21 21:32:33 +00:00
Yujong Lee
a5571333fb refactor(cache-valkey-semantic): reuse cache-redis connection layer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:25:22 +00:00
Yujong Lee
3ce436af5e refactor(cache-disk): isolate python compatibility behind a value adapter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:22:22 +00:00
Yujong Lee
4509eb9914 fix(cache): align native semantic cache scope keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:22:01 +00:00
Yujong Lee
5acfcfa4fe feat(rust): native Azure Blob response cache backend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 21:00:48 +00:00
Yujong Lee
83d4ce05c9 feat(rust-cache): serve RedisClusterCache natively as a Redis topology
Extend cache-redis so RedisTopology::Cluster routes single-key commands by
hash slot, groups pipelines by slot while keeping reply order, scans and
scoped-flushes every primary, and fans admin commands out to all nodes.
The bridge projects RedisClusterCache startup_nodes into the typed topology,
accepts the exact RedisClusterCache identity, guards nodes_manager state,
and falls back to Python for anything it cannot project.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:49:21 +00:00
Yujong Lee
605729a194 fix(rust): pin aws-sdk-s3 without relaxing eventstream pins
Keep the exact eventstream pins but move them to =0.61.4 so aws-runtime 1.9.4's eventstream requirement resolves, downgrade aws-sdk-s3 to 1.146.1, and echo the context ttl from S3Cache::get_ttl to match Python

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 20:39:17 +00:00