build(redis-resolve): add the script that regenerates redis-py-kwargs.json

redis_py_kwargs.py reads each redis-py version's signatures in a throwaway uv
environment and the REDIS_* names from litellm._redis under the locked
version, then writes the catalog. Run on this branch it reproduces every arg row
of the hand-made file, so only the meta text changes.

test_redis.py now loads the script's arg_signatures instead of keeping its own
copy of the introspection, so the code that writes the file is the code that
checks it in every redis-compat CI job
This commit is contained in:
Yujong Lee 2026-11-05 12:51:30 +09:00
parent d664e21c57
commit 6d800a887b
4 changed files with 129 additions and 29 deletions

View file

@ -4,7 +4,7 @@
- The `Resolution` holds the target, the auth and the timeouts Rust models, every other plain setting verbatim in `passthrough`, the names of object-valued settings in `host_objects`, and every setting the Rust client ignores as an `Ignored` with its source and consequence, which the host logs. A host passes its `credential_provider` object as a `ProviderHandle`, and a `credential_provider` key without one is `Error::ProviderWithoutHandle`. Azure's username is read from `os.environ` by Python, so `interpret` leaves `AzureAdAuth::username` unset and the host fills it in
- `options.rs` reads the kwargs into the typed `Options` and splits the unmodeled settings into passthrough and host objects and collects ignored ones. `endpoint.rs`, `auth.rs` and `tls.rs` pick the target, the credentials and TLS from `Options` alone
- Rust never introspects redis-py. Every Redis kwarg redis-py or `litellm/_redis.py` accepts is one row in `SETTINGS` in `src/keys.rs`, and that row is the whole answer for the kwarg: its `Handling` (`Read`, `Warn` with the consequence the warning names, or `Reject`), its `Shape` (plain data lands in `passthrough`, a Python object in `host_objects`), whether a url query carries it (`FromUrl`), whether `_get_redis_env_kwarg_mapping` reads it as `REDIS_<KWARG>`, and the names `_merge_redis_kwargs` falls back to. The last two columns are only read by the gateway merge, and they stay on the row so there is one table. A `Read` row is a field of `Options`, which an inline test pins. A kwarg with no row is not a Redis setting, so it passes through without a warning
- `redis-py-kwargs.json` holds the Python facts each row answers: which redis-py versions name the kwarg in which signature, and whether Python reads its `REDIS_*` name under the locked redis-py. `test_redis_py_kwargs_catalog_matches_the_installed_redis_py` in `tests/unit/test_redis.py` checks it against each version the `redis-compat` CI matrix installs, and `tests/catalog.rs` fails until `SETTINGS` has exactly one row per kwarg in it with the same `environment`. A redis-py bump that adds a kwarg therefore cannot land without a row deciding how Rust handles it
- `redis-py-kwargs.json` holds the Python facts each row answers: which redis-py versions name the kwarg in which signature, and whether Python reads its `REDIS_*` name under the locked redis-py. Never edit it by hand. `redis_py_kwargs.py` next to it regenerates it, and its docstring says how to run it. A kwarg litellm reads that no signature or `REDIS_*` name covers goes in its `LITELLM_ONLY`, and a new redis-py version in its `VERSIONS` and the `redis-compat` CI matrix together. `test_redis_py_kwargs_catalog_matches_the_installed_redis_py` in `tests/unit/test_redis.py` checks it against each version the `redis-compat` CI matrix installs, and `tests/catalog.rs` fails until `SETTINGS` has exactly one row per kwarg in it with the same `environment`. A redis-py bump that adds a kwarg therefore cannot land without a row deciding how Rust handles it
- The contract is `resolves_like_python_redis` in `tests/interpret.rs` and its mirror `test_redis_settings_resolve_like_the_rust_resolver_expects` in `tests/unit/test_redis.py`. Each case holds the merged dict, the sources the merge gave it, and the expected view written next to it. The Python side asserts that `_merge_redis_kwargs` returns that merged dict and reads the effective redis-py 5.3.1 connection objects without connecting, and `merges_like_python_redis` in `gateway/src/redis/surface_tests.rs` pins the same merged dicts from the Rust merge. A new rule is a new case in every file, never a generated fixture. Python and Rust pods share one Redis during the switch, so a change here is a change to a wire contract
- A url carries its own TLS. redis-py's url allowlist is the plain `Connection` signature, so every `ssl_*` setting outside the url is dropped, and a credential provider strips the url's userinfo. A `rediss://` url's `ssl_cert_reqs`, `ssl_ca_certs`, `ssl_ca_data`, `ssl_ca_path`, `ssl_certfile`, `ssl_keyfile`, `ssl_password` and `ssl_check_hostname` query parameters are read like `parse_url` reads them
- TLS options and `db` are only read for the endpoint that uses them, so a bad `ssl_cert_reqs` never fails a url target. A cluster with a non-zero `db`, and Sentinel with a `host` or `port`, are errors because redis-py raises building those clients

View file

@ -1,16 +1,14 @@
{
"meta": {
"description": "What redis-py and litellm/_redis.py accept for every Redis kwarg the Rust resolver can be handed. SETTINGS in src/keys.rs has exactly one row per arg here",
"generated": "2026-11-05",
"redis_py_versions": [
"5.3.1",
"6.4.0",
"7.4.1",
"8.0.1"
],
"redis_py_locked": "5.3.1",
"python": "3.12",
"method": "uv run --no-project --python 3.12 --with redis==<version> python -I, reading inspect.getfullargspec(inspect.unwrap(cls.__init__)) args and kwonlyargs, minus self",
"method": "python redis_py_kwargs.py next to this file, see its docstring. It reads inspect.getfullargspec(inspect.unwrap(cls.__init__)) args and kwonlyargs, minus self",
"signatures": {
"sync": "redis.Redis.__init__. _get_redis_kwargs uses it as the config allowlist and the REDIS_<KWARG> environment mapping",
"async": "redis.asyncio.Redis.__init__, which the proxy constructs. A kwarg in sync but not async raises TypeError when the client is built",
@ -19,7 +17,8 @@
},
"environment": "true when _get_redis_env_kwarg_mapping reads the arg as REDIS_<ARG> under the locked redis-py",
"redis_py": "each redis-py version whose signatures name the arg, with which ones. An empty object means only litellm reads it",
"checked_by": "test_redis_py_kwargs_catalog_matches_the_installed_redis_py in tests/unit/test_redis.py checks the locked version against the installed redis-py, and tests/catalog.rs checks SETTINGS against this file"
"checked_by": "tests/unit/test_redis.py checks the installed redis-py's column and the REDIS_* names against this file, and tests/catalog.rs checks SETTINGS against it",
"redis_py_locked": "5.3.1"
},
"args": {
"address_remap": {"environment": false, "redis_py": {"5.3.1": ["cluster"], "6.4.0": ["cluster"], "7.4.1": ["cluster"], "8.0.1": ["cluster"]}},

View file

@ -0,0 +1,110 @@
"""Regenerates redis-py-kwargs.json, the Python facts SETTINGS in src/keys.rs answers.
Run it from the repository root in the project environment, which has the locked redis-py and litellm:
uv run python litellm-rust/crates/redis-resolve/redis_py_kwargs.py
It reads every other redis-py version's signatures in a throwaway environment through
`uv run --no-project --with redis==<version>`, so nothing else is installed. Then update SETTINGS until
tests/catalog.rs passes.
"""
import inspect
import json
import subprocess
import sys
from collections.abc import Mapping
from pathlib import Path
from typing import Final
VERSIONS: Final = ("5.3.1", "6.4.0", "7.4.1", "8.0.1")
PYTHON: Final = "3.12"
CATALOG: Final = Path(__file__).with_name("redis-py-kwargs.json")
# Keys litellm/_redis.py or its callers read that neither a redis-py signature nor the REDIS_* mapping names
LITELLM_ONLY: Final = frozenset(
{"connection_class", "namespace", "sentinel_nodes", "sentinel_password", "service_name"}
)
META: Final = {
"description": "What redis-py and litellm/_redis.py accept for every Redis kwarg the Rust resolver can be handed. SETTINGS in src/keys.rs has exactly one row per arg here",
"redis_py_versions": list(VERSIONS),
"python": PYTHON,
"method": "python redis_py_kwargs.py next to this file, see its docstring. It reads inspect.getfullargspec(inspect.unwrap(cls.__init__)) args and kwonlyargs, minus self",
"signatures": {
"sync": "redis.Redis.__init__. _get_redis_kwargs uses it as the config allowlist and the REDIS_<KWARG> environment mapping",
"async": "redis.asyncio.Redis.__init__, which the proxy constructs. A kwarg in sync but not async raises TypeError when the client is built",
"connection": "redis.Connection or redis.asyncio.Connection MRO, the url allowlist from _get_redis_url_kwargs",
"cluster": "redis.asyncio.RedisCluster.__init__, the allowlist from _get_redis_cluster_kwargs",
},
"environment": "true when _get_redis_env_kwarg_mapping reads the arg as REDIS_<ARG> under the locked redis-py",
"redis_py": "each redis-py version whose signatures name the arg, with which ones. An empty object means only litellm reads it",
"checked_by": "tests/unit/test_redis.py checks the installed redis-py's column and the REDIS_* names against this file, and tests/catalog.rs checks SETTINGS against it",
}
def _named_args(*classes: type) -> frozenset[str]:
return frozenset(
name
for cls in classes
for spec in (inspect.getfullargspec(inspect.unwrap(cls.__init__)),)
for name in spec.args + spec.kwonlyargs
if name != "self"
)
def arg_signatures() -> dict[str, list[str]]:
"""Each kwarg the installed redis-py names, with the signatures that name it."""
import redis
import redis.asyncio as async_redis
connection_classes: Final = (*redis.Connection.__mro__, *async_redis.Connection.__mro__)
signatures: Final = {
"sync": _named_args(redis.Redis),
"async": _named_args(async_redis.Redis),
"connection": _named_args(*(cls for cls in connection_classes if cls is not object)),
"cluster": _named_args(async_redis.RedisCluster),
}
names: Final = sorted(frozenset().union(*signatures.values()))
return {name: [kind for kind, named in signatures.items() if name in named] for name in names}
def _arg_signatures_of(version: str) -> Mapping[str, list[str]]:
command: Final = ("uv", "run", "--no-project", "--python", PYTHON, "--with", f"redis=={version}")
script: Final = ("python", "-I", __file__, "--arg-signatures")
return json.loads(subprocess.run((*command, *script), check=True, capture_output=True, text=True).stdout)
def catalog() -> dict[str, object]:
import redis
from litellm._redis import _get_redis_env_kwarg_mapping
environment: Final = frozenset(_get_redis_env_kwarg_mapping().values())
by_version: Final = {version: _arg_signatures_of(version) for version in VERSIONS}
names: Final = sorted(LITELLM_ONLY | environment | frozenset().union(*by_version.values()))
return {
"meta": {**META, "redis_py_locked": redis.__version__},
"args": {
name: {
"environment": name in environment,
"redis_py": {version: args[name] for version, args in by_version.items() if name in args},
}
for name in names
},
}
# one line per arg, so a version bump reads as a diff of the args it touched
def render(catalog: Mapping[str, object]) -> str:
meta: Final = json.dumps(catalog["meta"], indent=2).replace("\n", "\n ")
args: Final = catalog["args"]
assert isinstance(args, dict)
rows: Final = ",\n".join(f" {json.dumps(name)}: {json.dumps(arg)}" for name, arg in args.items())
return f'{{\n "meta": {meta},\n "args": {{\n{rows}\n }}\n}}\n'
if __name__ == "__main__":
if sys.argv[1:] == ["--arg-signatures"]:
sys.stdout.write(json.dumps(arg_signatures()))
else:
CATALOG.write_text(render(catalog()))

View file

@ -1,9 +1,10 @@
import importlib.util
import inspect
import json
import os
import ssl
from pathlib import Path
from types import SimpleNamespace
from types import ModuleType, SimpleNamespace
from typing import Final
from unittest.mock import AsyncMock, MagicMock, patch
@ -3264,37 +3265,27 @@ def test_redis_settings_resolve_like_the_rust_resolver_expects(monkeypatch, kwar
assert outcome == expected
_REDIS_PY_KWARGS: Final = json.loads(
(Path(__file__).parents[2] / "litellm-rust/crates/redis-resolve/redis-py-kwargs.json").read_text()
)
_REDIS_RESOLVE: Final = Path(__file__).parents[2] / "litellm-rust/crates/redis-resolve"
_REDIS_PY_KWARGS: Final = json.loads((_REDIS_RESOLVE / "redis-py-kwargs.json").read_text())
def _named_args(*classes: type) -> frozenset[str]:
return frozenset(
name
for cls in classes
for spec in (inspect.getfullargspec(inspect.unwrap(cls.__init__)),)
for name in spec.args + spec.kwonlyargs
if name != "self"
)
def _catalog_script() -> ModuleType:
spec: Final = importlib.util.spec_from_file_location("redis_py_kwargs", _REDIS_RESOLVE / "redis_py_kwargs.py")
assert spec is not None and spec.loader is not None
module: Final = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def test_redis_py_kwargs_catalog_matches_the_installed_redis_py():
assert redis.__version__ in _REDIS_PY_KWARGS["meta"]["redis_py_versions"]
signatures: Final = {
"sync": _named_args(redis.Redis),
"async": _named_args(async_redis.Redis),
"connection": _named_args(
*(cls for cls in (*redis.Connection.__mro__, *async_redis.Connection.__mro__) if cls is not object)
),
"cluster": _named_args(async_redis.RedisCluster),
catalogued: Final = {
name: arg["redis_py"][redis.__version__]
for name, arg in _REDIS_PY_KWARGS["args"].items()
if redis.__version__ in arg["redis_py"]
}
args: Final = _REDIS_PY_KWARGS["args"]
names: Final = frozenset(args).union(*signatures.values())
catalogued: Final = {name: args.get(name, {}).get("redis_py", {}).get(redis.__version__, []) for name in names}
installed: Final = {name: [kind for kind, named in signatures.items() if name in named] for name in names}
assert catalogued == installed
assert catalogued == _catalog_script().arg_signatures()
@pytest.mark.skipif(