litellm/litellm-rust/crates/redis-store/AGENTS.md

• One folder per Redis family (router_cooldowns/, router_usage/, spend_counters/, auth_cache/, rate_limits/), each with mod.rs (the store and its ops), keys.rs (key types and spellings) and types.rs (the family's values, with the FromValue / IntoValue codecs for what it stores in Redis). It is the only place that family's keys, value codec and scripts are written, and it takes domain ids, never keys. Shared pieces live once at the crate root: Error and ErrorKind (error.rs), Memory and Clock (memory.rs), Pending (request.rs), Remote and its attach-once RemoteSlot, which knows nothing of memory (remote.rs), DualCache (dual_cache/) and the Python codecs (codec.rs, value.rs)
• An op declared on a Send returns Pending<T>, a lazy future built when the op is declared. An op that only means something on Redis (an MGET the caller falls back from, a script) returns None without one, and an op that replaced a DualCache call answers from memory. A post-call method returns false when there was no Redis to declare on
• A store that sends a pipeline of its own uses RedisBatch::pipelined, never Pipeline and raw Values
• Python and Rust pods share these keys during a rolling deploy, so a module writes the bytes main's Python writes and reads everything main reads. Python formats go through litellm-python-compat (repr, literal_eval, json), never a hand-rolled codec
• A family holds one DualCache per DualCache its Python held on main, and each DualCache method is the Python method it names, so a family op that was a DualCache call is one call here. Where main's Python reached past DualCache to in_memory_cache or redis_cache, the family does the same through memory() and redis(), which keeps that reach visible. Caches a family's Python attached to one RedisCache share one Arc<RemoteSlot>. Only a family that batch-reads builds its cache with DualCache::batch_reading from a TierPolicy the host passes in, and the Python host fills it from what main reads from the environment or litellm globals (litellm/rust_bridge/host/settings.py, router_tiers). Callers pass a Remote (or none), never a cache. Redis-only families are plain functions over &RedisClient
• Every family works with no Redis, answering from memory. attach adds the Redis tier at most once and keeps memory, as DualCache.attach_redis_cache does, so a host builds the store at startup and never rebuilds it to connect
• Where main swallows a Redis failure and serves memory, the family does the same and logs. It never turns a failure into a miss that main would not
• Key suffixes and wire spellings derive from strum on the family's enums. A value the family stores in Redis implements FromValue / IntoValue over litellm_python_compat::Value. An argument or reply is a typed record, never a positional tuple: a frozen Python dataclass read in python-bridge by a #[derive(FromPyObject)] struct with #[pyo3(attribute)] fields and converted into the family's type with From, and a reply returned as a frozen #[pyclass]. The bridge reads an enum through its strum spelling, never a hand-written match
• Wire contracts are hand-written cases with expected values, inlined as the same literals in this crate's test and in tests/unit/redis/stores/, checked here against real Redis and there against Python's own codec. Change both sides together
• rate_limits.rs keeps the v3 limiter's Lua byte for byte (rate_limits/lua.rs) so pods on either stack share one script SHA; the admission decisions stay in the Python limiter
• RequestRedis is one request's batches: a pre-call and a post-call RedisBatch per client, the post one flushed by flush_post or its deadline. Each op exists once and takes a Send (Now, PreCall, PostCall) picked by the caller, and the store calls send.batch(&client). A store never encodes a phase in a method name. A post-call method exists only where a follow-up runs after the write settles (token_increment_post_call), as a thin wrapper over the shared op. Send::Now is an alone batch, so a script call outside a request goes out alone, like a redis-py Script outside a pipeline, so NOSCRIPT costs SCRIPT LOAD and one more EVALSHA the way it does on main. Only what main's Python puts on its request pipeline rides these batches. An op main sends on its own round trip, like a deployment's tpm and rpm increment, stays on its own round trip here
• Every store sends the commands main's RedisCache sends for that operation: MGET where Python reads several keys at once, SET ... EX, and INCRBYFLOAT with EXPIRE or TTL. A MONITOR diff against main is a parity check, and any difference is a defect
• Tests run against litellm-redis-testing on standalone and cluster. A test is single-topology only for outages