mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-13 08:52:02 +00:00
Reworks the redis-client, connect, resolve and testing crates, adds python-compat json loads and TTL seconds, replaces the redis-store memory tier with a DualCache plus a shared failure log, and gives the Python side the Redis timeout log throttle and a request OTEL parent span for service events Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
11 KiB
11 KiB
Interpreting merged kwargs
- The boundary is merged redis-py kwargs in,
litellm_redis_types::Resolutionout.interprettakes the dict a host already merged the way_merge_redis_kwargsinlitellm/_redis.pydoes, with itsREDIS_*values andget_secretfallbacks filled, and stands in for redis-py's constructors andfrom_url. It never reads the environment or a secret, so both hosts share it. EveryEntrykeeps the source a warning or error names, whichkwargssets to the kwarg name and the gateway merge sets toREDIS_<KWARG>or the fallback name - The
Resolutionholds the target, the auth and the timeouts Rust models, and every setting the Rust client ignores as anIgnoredwith its source and consequence, which the host logs. A host passes itscredential_providerobject as aProviderHandle, and acredential_providerkey without one isError::ProviderWithoutHandle. Python's Azure AD auth readsREDIS_USERNAMEfromos.environinstead of the merged kwargs, so it is a required argument ofinterpretthat each host reads from its own process environment, the gateway through itsLookupand python-bridge through Python'sos.environ. An empty one is unset, likeor None options.rsreads the kwargs into the typedOptionsand collects the ignored ones.endpoint.rs,auth.rsandtls.rspick the target, the credentials and TLS fromOptionsalone- Rust never introspects redis-py. Every Redis kwarg redis-py or
litellm/_redis.pyaccepts is one row inSETTINGSinsrc/keys.rs, and that row is the whole answer for the kwarg: itsHandling(Read,Warnwith the consequence the warning names, orReject), whether a url query carries it (FromUrl), whether_get_redis_env_kwarg_mappingreads it asREDIS_<KWARG>, and the names_merge_redis_kwargsfalls back to. The last two columns are only read by the gateway merge, and they stay on the row so there is one table. AReadrow is a field ofOptions, which an inline test pins. A kwarg with no row is not a Redis setting, so it is dropped without a warning redis-py-kwargs.jsonholds the Python facts each row answers: which redis-py versions name the kwarg in which signature, and whether Python reads itsREDIS_*name under the locked redis-py. Never edit it by hand.redis_py_kwargs.pynext to it regenerates it, and its docstring says how to run it. A kwarg litellm reads that no signature orREDIS_*name covers goes in itsLITELLM_ONLY, and a new redis-py version in itsVERSIONSand theredis-compatCI matrix together.test_redis_py_kwargs_catalog_matches_the_installed_redis_pyintests/unit/test_redis.pychecks it against each version theredis-compatCI matrix installs, andtests/catalog.rsfails untilSETTINGShas exactly one row per kwarg in it with the sameenvironment. A redis-py bump that adds a kwarg therefore cannot land without a row deciding how Rust handles it- The contract is
resolves_like_python_redisintests/interpret.rsand its mirrortest_redis_settings_resolve_like_the_rust_resolver_expectsintests/unit/test_redis.py. Each case holds the merged dict, the sources the merge gave it, theREDIS_USERNAMEthe host passes, and the expected outcome written next to it.Resolvesis the view both sides build.Refusesis a config Rust refuses and Python connects with, a disagreement on purpose.Failsis a config redis-py 5.3.1 already raises on, building the client or its first connection, so Rust failing changes no deployment that works today. Its Rust side names the classError::raised_building_the_python_clientanswers,Nonewhen Python only raises once a command connects, and its Python mirror_Failsasserts the class and message redis-py raises. The Python side asserts that_merge_redis_kwargsreturns that merged dict and reads the effective redis-py 5.3.1 connection objects without connecting, andmerges_like_python_redisingateway/src/redis/surface_tests.rspins the same merged dicts from the Rust merge. A new rule is a new case in every file, never a generated fixture. Python and Rust pods share one Redis during the switch, so a change here is a change to a wire contract - A url carries its own TLS. redis-py's url allowlist is the plain
Connectionsignature, so everyssl_*setting outside the url is dropped, and a credential provider strips the url's userinfo. Arediss://url'sssl_cert_reqs,ssl_ca_certs,ssl_ca_data,ssl_ca_path,ssl_certfile,ssl_keyfile,ssl_passwordandssl_check_hostnamequery parameters are read likeparse_urlreads them - TLS options and
dbare only read for the endpoint that uses them, so a badssl_cert_reqsnever fails a url target. A cluster with a non-zerodb, and Sentinel with ahostorport, are errors because redis-py raises building those clients - An empty value (
"",[],{}) is the same as an unset one, like Python'sx or .... A null is unset too, exceptssl_cert_reqs, where redis-py readsNoneasCERT_NONE, so a null skips certificate verification ssl_cert_reqstakesnone,optionalandrequiredcase-sensitively like redis-py's lookup, soNone(whatREDIS_SSL_CERT_REQS=Nonegives) isError::Invalid, and anssl.VerifyModeint, where 0 skips verification and 1 or 2 verifies.optionalverifies, because rustls has no optional mode- Precedence matches Python. The target is cluster, then URL, then Sentinel (
sentinel_nodesplusservice_name), then host. Auth is acredential_providerhandle, then GCP, then Azure, then AWS, and the first one wins with a warning - Numbers are read with
litellm_python_compat::number, so whitespace and digit underscores parse like Python'sint()andfloat(). A value they cannot parse (port: abc,socket_timeout: slow) is dropped with a warning and the default is used, like_coerce_redis_kwargs_types. A number that parses but is out of range (negative seconds, a port above 65535) fails withError::Invalid - Flags (
ssl,ssl_check_hostname,aws_iam_auth,aws_iam_serverless) are read one way on every surface.true,1andyesare true andfalse,0,no,off,fandnare false, the text redis-py's_str_to_booland the proxy's pydantic models agree on. Any other text isError::Invalid, so a TLS setting is never guessed - The resolver fails on a setting it cannot honor only when that setting is a security check. A
Warnrow is dropped and returned as anIgnorednaming its source (ssl_ciphersorREDIS_SSL_CIPHERS) and the consequence on its row, because a Python deployment that worked must keep working when its stores move to Rust. Only a value that is malformed (Error::InvalidorError::UrlQuery), a target that is missing or aRejectrow that asks for its check fails. Promoting aWarnrow toReadis then a pure improvement with no behavior cliff, while moving one toRejectcan fail a config that connects today- A
Warnrow with ametcheck is dropped silently when its value asks for nothing more than Rust does, sossl_min_versionof TLS 1.2 or lower never warns - A
Rejectrow is a security check rustls cannot perform that Python's locked async client already refuses with aTypeError, so failing closed breaks no proxy that runs today.tests/catalog.rsfails if the locked redis-py's async, cluster or connection signature accepts aRejectrow, so a redis-py bump forces the decision again. A row'sAskedsays which values ask for the check, from the kwargs,REDIS_*or a url query, and those areError::Unenforceablessl_validate_ocspandssl_validate_ocsp_stapledask when true, because rustls has no client-side OCSP. False asks for nothing and is dropped without a warningssl_include_verify_flagsandssl_exclude_verify_flagsask when set, because rustls takes no X.509 verify flags such as CRL checking. An empty list asks for nothing
ssl_ca_dataandssl_ca_pathare read intoTlsOptionsand trusted next tossl_ca_certs.ssl_passwordis read intoTlsOptions::key_password, whichredis-clientuses to decrypt a PKCS#8 key like the sync client'sload_cert_chain(password=...). redis-py 5.3.1's async client refuses it, so this is a superset of the path the proxy uses, likessl_ca_pathunix_socket_pathresolves toTarget::Unix, which like redis-py'sUnixDomainSocketConnectionreplaces host, port and TLS. Python still requireshostnext to it, so Rust does too- A url's
?db=,?username=,?password=,?socket_timeout=,?socket_connect_timeout=and?health_check_interval=are read likeparse_urlreads them, so?db=wins over the path, the userinfo wins over?username=and?password=, and the timeouts and health check win over the kwargs likefrom_url. A repeated parameter takes its first value likeparse_qs. A numberint()orfloat()cannot read isError::UrlQuery, asparse_urlraisesValueError. Any other query parameter, including one with no row, is ignored namingurl ?<name>, because redis-py turns each one into a client kwarg. Which ones a url carries is theFromUrlcolumn socket_keepaliveis a flag that defaults to true on every target, like_get_redis_cluster_kwargsdefaults it for a cluster. Python leaves it off for a standalone or Sentinel client, so an unset value there is a difference on purpose, because without keepalive a load balancer drops an idle connection silently. False turns it off.socket_keepalive_optionsis warned, andredis-clientprobes after its own idle time insteadhealth_check_intervalis read as anintof seconds like_coerce_redis_kwargs_typesintoHealthCheck. Zero isDisabledand stays off on every target, a negative interval checks before every command like redis-py, and onlyUnsettakes a default.redis-connectgives an unset cluster Python's 25s default andredis-clientpings every node before the next command once the client sat idle that long, outside the breaker and the call events
- A
Error::raised_building_the_python_clientis how python-bridge treats a config Rust cannot use. Where Python raises building its client, the bridge raises the same class fromNativeRedisClient.connect. Where Python builds the client and only fails once a command connects, whichRedisCachecatches and logs on every call, the stores stay on memory and the bridge logs an error- Cases where Rust and Python disagree on purpose are cases in both contract tests with a different expected outcome on each side
ssl_ca_path: redis-py 5.3.1's asyncRedishas no such parameter and raisesTypeErrorbuilding the client, while Rust trusts the directory likeload_verify_locations(capath=...)on,tandyfor a flag areError::Invalidon every surface. pydantic reads them as true forcoordination_redisand_str_to_boolreads them as false everywhere else, so Rust refuses to pick one. Other unrecognized text is false in Python and an error heressl_cert_reqsas anssl.VerifyModeint: redis-py 5.3.1's sync client takes it, while its asyncRedisSSLContextleavescert_reqsunset and raisesAttributeErroronce a command connects. Rust reads it like the sync client- Empty
sentinel_nodeswith noREDIS_SENTINEL_NODESis no target at all, so it fails as "either host or url must be specified". redis-py keeps the empty list and fails building the Sentinel client