litellm/litellm-rust/crates/redis-resolve/AGENTS.md
yujonglee a1b87bcca8
feat(redis): extend the native Redis foundation for the store families (#50169)
Reworks the redis-client, connect, resolve and testing crates, adds python-compat json loads and
TTL seconds, replaces the redis-store memory tier with a DualCache plus a shared failure log, and
gives the Python side the Redis timeout log throttle and a request OTEL parent span for service events

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 12:30:07 +09:00

11 KiB

Interpreting merged kwargs

  • The boundary is merged redis-py kwargs in, litellm_redis_types::Resolution out. interpret takes the dict a host already merged the way _merge_redis_kwargs in litellm/_redis.py does, with its REDIS_* values and get_secret fallbacks filled, and stands in for redis-py's constructors and from_url. It never reads the environment or a secret, so both hosts share it. Every Entry keeps the source a warning or error names, which kwargs sets to the kwarg name and the gateway merge sets to REDIS_<KWARG> or the fallback name
  • The Resolution holds the target, the auth and the timeouts Rust models, and every setting the Rust client ignores as an Ignored with its source and consequence, which the host logs. A host passes its credential_provider object as a ProviderHandle, and a credential_provider key without one is Error::ProviderWithoutHandle. Python's Azure AD auth reads REDIS_USERNAME from os.environ instead of the merged kwargs, so it is a required argument of interpret that each host reads from its own process environment, the gateway through its Lookup and python-bridge through Python's os.environ. An empty one is unset, like or None
  • options.rs reads the kwargs into the typed Options and collects the ignored ones. endpoint.rs, auth.rs and tls.rs pick the target, the credentials and TLS from Options alone
  • Rust never introspects redis-py. Every Redis kwarg redis-py or litellm/_redis.py accepts is one row in SETTINGS in src/keys.rs, and that row is the whole answer for the kwarg: its Handling (Read, Warn with the consequence the warning names, or Reject), whether a url query carries it (FromUrl), whether _get_redis_env_kwarg_mapping reads it as REDIS_<KWARG>, and the names _merge_redis_kwargs falls back to. The last two columns are only read by the gateway merge, and they stay on the row so there is one table. A Read row is a field of Options, which an inline test pins. A kwarg with no row is not a Redis setting, so it is dropped without a warning
  • redis-py-kwargs.json holds the Python facts each row answers: which redis-py versions name the kwarg in which signature, and whether Python reads its REDIS_* name under the locked redis-py. Never edit it by hand. redis_py_kwargs.py next to it regenerates it, and its docstring says how to run it. A kwarg litellm reads that no signature or REDIS_* name covers goes in its LITELLM_ONLY, and a new redis-py version in its VERSIONS and the redis-compat CI matrix together. test_redis_py_kwargs_catalog_matches_the_installed_redis_py in tests/unit/test_redis.py checks it against each version the redis-compat CI matrix installs, and tests/catalog.rs fails until SETTINGS has exactly one row per kwarg in it with the same environment. A redis-py bump that adds a kwarg therefore cannot land without a row deciding how Rust handles it
  • The contract is resolves_like_python_redis in tests/interpret.rs and its mirror test_redis_settings_resolve_like_the_rust_resolver_expects in tests/unit/test_redis.py. Each case holds the merged dict, the sources the merge gave it, the REDIS_USERNAME the host passes, and the expected outcome written next to it. Resolves is the view both sides build. Refuses is a config Rust refuses and Python connects with, a disagreement on purpose. Fails is a config redis-py 5.3.1 already raises on, building the client or its first connection, so Rust failing changes no deployment that works today. Its Rust side names the class Error::raised_building_the_python_client answers, None when Python only raises once a command connects, and its Python mirror _Fails asserts the class and message redis-py raises. The Python side asserts that _merge_redis_kwargs returns that merged dict and reads the effective redis-py 5.3.1 connection objects without connecting, and merges_like_python_redis in gateway/src/redis/surface_tests.rs pins the same merged dicts from the Rust merge. A new rule is a new case in every file, never a generated fixture. Python and Rust pods share one Redis during the switch, so a change here is a change to a wire contract
  • A url carries its own TLS. redis-py's url allowlist is the plain Connection signature, so every ssl_* setting outside the url is dropped, and a credential provider strips the url's userinfo. A rediss:// url's ssl_cert_reqs, ssl_ca_certs, ssl_ca_data, ssl_ca_path, ssl_certfile, ssl_keyfile, ssl_password and ssl_check_hostname query parameters are read like parse_url reads them
  • TLS options and db are only read for the endpoint that uses them, so a bad ssl_cert_reqs never fails a url target. A cluster with a non-zero db, and Sentinel with a host or port, are errors because redis-py raises building those clients
  • An empty value ("", [], {}) is the same as an unset one, like Python's x or .... A null is unset too, except ssl_cert_reqs, where redis-py reads None as CERT_NONE, so a null skips certificate verification
  • ssl_cert_reqs takes none, optional and required case-sensitively like redis-py's lookup, so None (what REDIS_SSL_CERT_REQS=None gives) is Error::Invalid, and an ssl.VerifyMode int, where 0 skips verification and 1 or 2 verifies. optional verifies, because rustls has no optional mode
  • Precedence matches Python. The target is cluster, then URL, then Sentinel (sentinel_nodes plus service_name), then host. Auth is a credential_provider handle, then GCP, then Azure, then AWS, and the first one wins with a warning
  • Numbers are read with litellm_python_compat::number, so whitespace and digit underscores parse like Python's int() and float(). A value they cannot parse (port: abc, socket_timeout: slow) is dropped with a warning and the default is used, like _coerce_redis_kwargs_types. A number that parses but is out of range (negative seconds, a port above 65535) fails with Error::Invalid
  • Flags (ssl, ssl_check_hostname, aws_iam_auth, aws_iam_serverless) are read one way on every surface. true, 1 and yes are true and false, 0, no, off, f and n are false, the text redis-py's _str_to_bool and the proxy's pydantic models agree on. Any other text is Error::Invalid, so a TLS setting is never guessed
  • The resolver fails on a setting it cannot honor only when that setting is a security check. A Warn row is dropped and returned as an Ignored naming its source (ssl_ciphers or REDIS_SSL_CIPHERS) and the consequence on its row, because a Python deployment that worked must keep working when its stores move to Rust. Only a value that is malformed (Error::Invalid or Error::UrlQuery), a target that is missing or a Reject row that asks for its check fails. Promoting a Warn row to Read is then a pure improvement with no behavior cliff, while moving one to Reject can fail a config that connects today
    • A Warn row with a met check is dropped silently when its value asks for nothing more than Rust does, so ssl_min_version of TLS 1.2 or lower never warns
    • A Reject row is a security check rustls cannot perform that Python's locked async client already refuses with a TypeError, so failing closed breaks no proxy that runs today. tests/catalog.rs fails if the locked redis-py's async, cluster or connection signature accepts a Reject row, so a redis-py bump forces the decision again. A row's Asked says which values ask for the check, from the kwargs, REDIS_* or a url query, and those are Error::Unenforceable
      • ssl_validate_ocsp and ssl_validate_ocsp_stapled ask when true, because rustls has no client-side OCSP. False asks for nothing and is dropped without a warning
      • ssl_include_verify_flags and ssl_exclude_verify_flags ask when set, because rustls takes no X.509 verify flags such as CRL checking. An empty list asks for nothing
    • ssl_ca_data and ssl_ca_path are read into TlsOptions and trusted next to ssl_ca_certs. ssl_password is read into TlsOptions::key_password, which redis-client uses to decrypt a PKCS#8 key like the sync client's load_cert_chain(password=...). redis-py 5.3.1's async client refuses it, so this is a superset of the path the proxy uses, like ssl_ca_path
    • unix_socket_path resolves to Target::Unix, which like redis-py's UnixDomainSocketConnection replaces host, port and TLS. Python still requires host next to it, so Rust does too
    • A url's ?db=, ?username=, ?password=, ?socket_timeout=, ?socket_connect_timeout= and ?health_check_interval= are read like parse_url reads them, so ?db= wins over the path, the userinfo wins over ?username= and ?password=, and the timeouts and health check win over the kwargs like from_url. A repeated parameter takes its first value like parse_qs. A number int() or float() cannot read is Error::UrlQuery, as parse_url raises ValueError. Any other query parameter, including one with no row, is ignored naming url ?<name>, because redis-py turns each one into a client kwarg. Which ones a url carries is the FromUrl column
    • socket_keepalive is a flag that defaults to true on every target, like _get_redis_cluster_kwargs defaults it for a cluster. Python leaves it off for a standalone or Sentinel client, so an unset value there is a difference on purpose, because without keepalive a load balancer drops an idle connection silently. False turns it off. socket_keepalive_options is warned, and redis-client probes after its own idle time instead
    • health_check_interval is read as an int of seconds like _coerce_redis_kwargs_types into HealthCheck. Zero is Disabled and stays off on every target, a negative interval checks before every command like redis-py, and only Unset takes a default. redis-connect gives an unset cluster Python's 25s default and redis-client pings every node before the next command once the client sat idle that long, outside the breaker and the call events
  • Error::raised_building_the_python_client is how python-bridge treats a config Rust cannot use. Where Python raises building its client, the bridge raises the same class from NativeRedisClient.connect. Where Python builds the client and only fails once a command connects, which RedisCache catches and logs on every call, the stores stay on memory and the bridge logs an error
  • Cases where Rust and Python disagree on purpose are cases in both contract tests with a different expected outcome on each side
    • ssl_ca_path: redis-py 5.3.1's async Redis has no such parameter and raises TypeError building the client, while Rust trusts the directory like load_verify_locations(capath=...)
    • on, t and y for a flag are Error::Invalid on every surface. pydantic reads them as true for coordination_redis and _str_to_bool reads them as false everywhere else, so Rust refuses to pick one. Other unrecognized text is false in Python and an error here
    • ssl_cert_reqs as an ssl.VerifyMode int: redis-py 5.3.1's sync client takes it, while its async RedisSSLContext leaves cert_reqs unset and raises AttributeError once a command connects. Rust reads it like the sync client
    • Empty sentinel_nodes with no REDIS_SENTINEL_NODES is no target at all, so it fails as "either host or url must be specified". redis-py keeps the empty list and fails building the Sentinel client