Commit graph

8 commits

Author SHA1 Message Date
yujonglee
574fcea691
fix(redis): share one native circuit breaker between RedisCache and the Rust stores (#50473)
* fix(redis): share one native circuit breaker between RedisCache and the Rust stores

RedisCache and the native Redis client each kept their own breaker, so once
the cache learned the coordination Redis was wedged, a spend-counter write
through the native store still paid a full socket timeout. Every Redis target
now has one Rust breaker, looked up by NativeRedisBreaker.shared and injected
into both the RedisCache guard and the native client. The Python FSM is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style(rust): cargo fmt the shared breaker

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(redis): close the breaker when the recovery probe gets a Redis reply error

A command error such as WRONGTYPE means Redis answered, but the probe was left
unsettled and its drop reopened the breaker for another recovery timeout. Both
the RedisCache guard and the native client now settle such a probe as a success.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(redis): pass a NativeRedisBreaker and isolate the shared breaker per test

The sensitive routing test still passed RedisBreakerSettings as the store breaker, and the session affinity fixture reused one Redis URL, so a breaker an earlier test opened leaked into the next. Also make the RUF043 match pattern raw.

* fix(redis): key the shared breaker by server and close the three audit gaps

The breaker registry now keys on the Redis server (host and port, unix socket,
cluster node set or sentinel service) instead of the full connection target, so
caches that differ only in db, credentials or url spelling share one breaker.

A READONLY reply from a replica mid-failover no longer counts as Redis answering,
so it can't reset a run of failures. A new RedisCache refreshes the breaker state
gauge, which the deleted Python breaker did at construction. A shared native client
is only reused by a store that passes the same breaker.

* refactor(redis): split redis-types into modules and drop redundant pyo3 attribute annotations

lib.rs now only declares modules and re-exports, with resolution, target, tls,
auth and endpoint each in their own file. Derived FromPyObject already reads a
struct field as the attribute of the same name, so #[pyo3(attribute)] on the
breaker and store settings did nothing.

* refactor(redis): drop redundant pyo3 attribute annotations in the stores and let PyO3 build the cooldown snapshot

Derived FromPyObject already reads each field as the attribute of the same name.
The cooldown snapshot returns an IndexMap, which PyO3 turns into the same dict in
the same order.

* refactor(redis): return a HashMap from the three memory-tier snapshots

The snapshots read cache-memory's HashMap, so there is no order to keep.
PyO3 builds the same dict from a std HashMap, which replaces the hand-written
PyDict loops in the spend counter and router usage stores and the IndexMap in
the cooldown store. The rate-limit slot registry keeps IndexMap because it
hands back a Python dict in the order Python wrote it.

* refactor(redis): share Redis breakers and clients through one RedisHosts per host

redis-connect's RedisHosts now owns the per-server breaker and the shared clients
that python-bridge kept in two process-wide statics. A client takes breaker
settings and admits through its server's breaker, so no caller can pair a client
with a breaker of its own, and clients are keyed by Resolution alone. This drops
RedisClient::guarded_by and Breaker::same.

python-bridge keeps one RedisHosts per process and calls it with the GIL released.
The gateway builds one per connect, so its coordination and response cache on the
same server now share a breaker. RedisStoreSettings.breaker is back to breaker
settings, as on main.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 17:50:06 +09:00
yujonglee
a1b87bcca8
feat(redis): extend the native Redis foundation for the store families (#50169)
Reworks the redis-client, connect, resolve and testing crates, adds python-compat json loads and
TTL seconds, replaces the redis-store memory tier with a DualCache plus a shared failure log, and
gives the Python side the Redis timeout log throttle and a request OTEL parent span for service events

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 12:30:07 +09:00
yujonglee
a1c238ff4a
refactor(rust): extract SecretValue and use it for Redis credentials (#49186)
* refactor(rust): move SecretValue into litellm-secret-value

* docs(rust): add rust-redaction skill

* refactor(redis-client): carry credentials as SecretValue

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-30 11:31:04 -07:00
yujonglee
bdfe7ae195
feat(gateway): connect to Redis from config.yaml (#48284)
* feat(gateway): connect to Redis from config.yaml

* feat(config): own Redis settings resolution

* fix(config): preserve Redis input precedence and presence

* fix(config): box Redis cache settings

* fix(gateway): preserve borrowed redis namespace

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 18:41:47 -07:00
yujonglee
0c0908c36e
feat(rust): add circuit-breaker crate and guard redis-client with it (#48277)
* feat(rust): add circuit-breaker crate and guard redis-client with it

* fix(circuit-breaker): ignore stale probe drops

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 18:41:47 -07:00
yujonglee
af0d29030e
feat(redis-client): run Lua scripts and pipelines with per-op errors (#48273)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 18:41:46 -07:00
yujonglee
a0b5c8b824
feat(redis-client): support Redis Cluster (#48271)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 18:41:46 -07:00
yujonglee
6edbd27f74
feat(rust): add redis-client crate for standalone Redis (#48269)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 18:41:46 -07:00