litellm/litellm-rust/crates/gateway/AGENTS.md

• Keep this crate a thin composition layer: mount endpoint routers and serve the supplied listener
• Server lifecycle and shared inbound middleware belong here, including client authentication, rate limiting, and request logging
• Endpoint paths, request handling, model resolution, and response encoding belong to the mounted crates. Provider execution belongs to the inference-<fmt> crates and llms
• Inject shared state and infrastructure. Avoid global runtimes, duplicate client pools, and abstractions for hypothetical endpoint groups
• Test mounting and server lifecycle through public HTTP behavior. Test endpoint semantics in the owning crate

Redis

• build_redis builds the Redis clients once at boot (src/redis/mod.rs). Components get a clone as a field, never a per-request extractor or a pool. redis-rs multiplexes one connection per node, so there is no connection pool and pool settings like max_connections are accepted and ignored
• Which Redis backs which role mirrors the proxy: coordination is general_settings.coordination_redis, else the cache_params Redis when cache: true and type: redis, else REDIS_* env. The transaction buffer reuses coordination, else env when use_redis_transaction_buffer is on. The response cache gets its own client so large values never queue in front of coordination commands
• src/redis/layers.rs is the Rust copy of _merge_redis_kwargs in litellm/_redis.py, which only this host needs because the Python host merges in Python. It puts the configured kwargs over their REDIS_* values and fills the get_secret fallbacks from the SETTINGS rows in redis-resolve, under the same conditions Python reads them, then hands the result to litellm_redis_resolve::interpret. Values come from a SecretSnapshot filled once at boot, read like get_secret (secret manager, then environment, with true and false read as bools), and the only straight environment read is Azure's REDIS_USERNAME
• A surface is a ConfiguredRedis constructor: coordination for general_settings.coordination_redis, which rejects a malformed port like the proxy's CoordinationRedisParams pydantic model, cache_params for litellm_settings.cache_params, which drops the cache-only keys and reads the redis_startup_nodes alias, and redis_cache_kwargs for a RedisCache.redis_kwargs dict in the contract tests. A new surface is a new constructor, never a flag
• Nulls depend on the surface. coordination_redis drops every null like model_dump(exclude_none=True), cache_params keeps a null as an explicit None that hides the REDIS_* value unless Cache or RedisCache names that parameter, and RedisCache.redis_kwargs keeps every null. A missing os.environ/ reference is a null, as load_config resolves it
• A setting the Rust client cannot honor warns and is dropped, and only a security check it cannot enforce fails boot, per redis-resolve/AGENTS.md
• Boot never waits for a configured Redis (clients connect lazily). Only the env-only coordination fallback is pinged, for 2s, and treated as absent when unreachable, as in Python
• /health/readiness never touches Redis, matching the proxy