mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-16 09:22:22 +00:00
Merge 2a964a58d0 into d23af8bdd0
This commit is contained in:
commit
7b36afa610
23 changed files with 2487 additions and 293 deletions
200
.agents/skills/redis-access/SKILL.md
Normal file
200
.agents/skills/redis-access/SKILL.md
Normal file
|
|
@ -0,0 +1,200 @@
|
|||
---
|
||||
name: redis-access
|
||||
description: Add Redis operations, move raw redis-py access or a keyspace behind RedisTransport and a store in litellm/redis, or explicitly port a store to Rust, without mixing the Python and Rust Redis stacks or expanding migration scope
|
||||
metadata:
|
||||
internal: true
|
||||
---
|
||||
|
||||
# Redis access
|
||||
|
||||
## Two stacks, never mixed
|
||||
|
||||
- Python keyspace code reaches Redis one way only, `Python store -> RedisPyTransport -> redis-py`
|
||||
- Rust keyspace code reaches Redis one way only, `Python caller -> pyo3 -> crates/redis-store -> crates/redis-client`, and the Rust gateway enters at `crates/redis-store`
|
||||
- A Python store never runs on the Rust client, and a Rust store never goes back through the Python transport. There is no Rust-backed `RedisTransport`, since that would be a second redis client binding over pyo3 that has to grow toward redis-py's whole command set
|
||||
- A keyspace is ported as a unit, keys plus codec plus Lua plus client, or not at all. Which stack a keyspace is on is decided per keyspace, never per call
|
||||
- During the transition a pod holds one redis-py pool and one Rust client to the same Redis, with one breaker each. That is correct, they guard different sockets. redis-py goes away only when the last keyspace, pub/sub and admin have a Rust home
|
||||
|
||||
## Adding new Redis access
|
||||
|
||||
- New coordination state gets a store in `litellm/redis/stores/<keyspace>.py`, not a new `RedisCache` method
|
||||
- A Python store is a finished state. Adding one does not require a Rust port
|
||||
- Callers get typed domain results from the store and keep policy, which TTL applies, fail open or closed, retries, and the `with_service_target` that labels the calls
|
||||
- New code never calls `init_async_client`, `init_pubsub_client`, `async_register_script` or `.redis_client`, and never imports redis-py outside `litellm/redis/transport/redis_py.py`. `litellm/caching/redis_cache.py` and `redis_cluster_cache.py` stay allowlisted while they own the breaker's exception types and the generic verbs
|
||||
|
||||
## The transport
|
||||
|
||||
`RedisTransport` in `litellm/redis/transport/contract.py` is how Python code reaches redis-py. It is a Protocol of plain methods, command-shaped and never redis-py shaped, and it covers every Redis use in Python, so a keyspace move never waits on the transport
|
||||
|
||||
- Data commands `get`, `mget`, `set`, `delete`, `incrby`, `incrbyfloat`, `expire`, `ttl`, `rpush`, `lpop`, `ltrim`, `sadd`, `scan` and `eval`
|
||||
- Batches, `batch()` for one caller's commands, `batch(transaction=True)` for MULTI/EXEC, and `request_batch()` / `post_call_batch()` for the request scope
|
||||
- Pub/sub, `publish`, `publish_all` and `subscribe`
|
||||
- Admin and health, `ping`, `info`, `client_list` and `flushall`
|
||||
|
||||
- Every method returns its value or a `RedisFailure`. Nothing raises for a Redis problem
|
||||
- `RedisFailure.kind` is `transport` (timeout or lost connection, the command may still have applied, so a failed write is an unknown outcome, never "not applied"), `server` (Redis answered with an error, the connection is fine and the breaker does not count it), `decode` (the reply was not the shape expected) or `breaker_open` (refused without touching the socket)
|
||||
- The namespace is applied once, inside the transport, to every key including Lua `KEYS` and pub/sub channels. A key that already starts with `{namespace}:` is left alone. Stores and callers pass logical keys
|
||||
- Every write takes its TTL from the caller as a `timedelta`. `None` means no expiry, there is no default
|
||||
- Replies are undecoded bytes. `eval` returns `int | bytes | None | tuple[...]`. Decoding belongs to the store that owns the keyspace
|
||||
- `eval` takes script text every call and does `EVALSHA` with `EVAL` fallback, so nothing registers scripts
|
||||
- These rules are shared with `crates/redis-client`, whose `AGENTS.md` has the Rust side. Change both together, since Python and Rust pods share keys in one Redis
|
||||
- `RedisCache.transport()` builds the transport over that cache's own client, pub/sub client, namespace, circuit breaker, `_admit_async_call`, cluster-aware `mget` and `ServiceLogging`. No new connection, no second breaker, and `RedisClusterCache`'s connection gate still applies
|
||||
- Every method runs its command through two steps, in this order. `run_under_breaker` wraps the existing `_run_under_circuit_breaker` from `litellm/caching/redis_cache.py`, so admission, the generation check, the health failure classification and the timeout flag are the ones legacy traffic uses, and maps the outcome to a `RedisFailure` (`breaker_open`, `transport`, `server`) instead of raising. `RedisServiceEvents.observe` times the call, logs a failure and emits the service event
|
||||
|
||||
- `ping`, the admin commands and pub/sub skip the breaker, like the `RedisCache` methods they replace, so health checks still reach a Redis the breaker took out. They still go through `RedisServiceEvents.observe`, except `get_message` polling
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RedisPyTransport:
|
||||
client: Callable[[], AsyncRedisCommands]
|
||||
namespace: str | None
|
||||
breaker: RedisCircuitBreaker
|
||||
admit: Admit
|
||||
events: RedisServiceEvents
|
||||
...
|
||||
|
||||
async def get(self, key: str) -> bytes | None | RedisFailure:
|
||||
return _bytes_reply(await self._run("get", lambda: self.client().get(self._key(key))))
|
||||
|
||||
async def _run(self, name: str, call: Callable[[], Awaitable[_T]]) -> _T | RedisFailure:
|
||||
return await self.events.observe(name, lambda: run_under_breaker(self.breaker, name, self.admit, call))
|
||||
```
|
||||
- A Redis command Python starts using is a method on the Protocol and on the batch, an implementation in `redis_py.py`, and a fakeredis test that the namespace is applied once, the TTL lands, a rejected command is `server` and an outage is `transport`
|
||||
- A store still picks the fewest round trips. A counter with a TTL on first write or an owner-checked release is one `eval`, not three commands
|
||||
- Multi-key commands and scripts keep their keys in one slot, or they break on cluster depending on slot ownership
|
||||
|
||||
## Batches
|
||||
|
||||
The pipeline engine is `PipelineBatch` in `litellm/redis/transport/batch.py`, the one `RedisBatch` has always had. Transport batches and `RedisBatch` (`litellm/caching/redis_batch.py`) declare onto it, so a request keeps one round trip per backend while callers move over one at a time
|
||||
|
||||
- Every batch method returns an awaitable reply, and awaiting any reply flushes everything declared so far. Each reply is its value or a `RedisFailure`, so one rejected command fails only its own reply, and an outage fails every reply and counts once on the breaker
|
||||
- `request_batch()` and `post_call_batch()` return the engine `request_redis_batch_scope` holds for this backend, or `None` outside a request. Never build a second request scope
|
||||
- A script Redis has not cached reruns on its own after the pipeline, so do not rely on its effect being visible to later commands in the same batch. A transaction batch sends `EVAL` instead, so its scripts run inside the MULTI
|
||||
- On a cluster client a request batch runs each command on its own, as `RedisBatch` always has. Explicit and transaction batches still pipeline
|
||||
- One REDIS event per flush, named by the batch and targeted by the key families that shared it, `mixed` with the sorted families when there are several
|
||||
|
||||
## Stores
|
||||
|
||||
A store owns one keyspace. Python and Rust pods share one Redis, so each keyspace needs one definition of its keys and values, and the store is that definition
|
||||
|
||||
- One module per keyspace. It defines a `Protocol` of domain operations named for what the caller wants (`acquire`, `release`, `increment`, `get`), never for Redis commands, and a frozen dataclass implementation that takes a `RedisTransport`
|
||||
- The store owns, and nothing outside it knows, the key builders (logical keys, no namespace), the value codec, expiry taken from the caller, the Lua, and reply decoding into typed results
|
||||
- Every operation returns its result or a `RedisFailure`. A value that does not decode is a `decode` failure, never a miss or a default
|
||||
- No logging, no retries, no fallback in a store. Those are policy and stay in the caller
|
||||
- A caller never builds a key, touches a client or runs a command for a keyspace that has a store. Exposing a key builder as a delegate for an existing public helper, or for the caller's in-memory fallback, is fine
|
||||
- Store operations take domain inputs, the ids and typed values the caller already has, and never a Redis key, a bundle of keys or encoded bytes. A native store implements the same `Protocol`, so anything the caller hands over ready-made stays in Python after a Rust port
|
||||
- The store encodes what it writes and decodes what it reads, both ways. A caller never serializes a value before handing it over, and never parses a value a store returned
|
||||
- A caller never calls a store's key builder to pass the result back into the store
|
||||
|
||||
```python
|
||||
# a caller building the key or the value
|
||||
await store.claim(f"mcp_gateway_dcr_code_used:{jti}", ttl=ttl)
|
||||
await store.acquire(refresh_lock_key(user_id, server_id), token, ttl=ttl)
|
||||
await store.save_record(batch_id, adapter.dump_json(reservation).decode(), ttl=ttl)
|
||||
|
||||
# the store owning both
|
||||
await store.claim(UsedCode(jti), ttl=ttl)
|
||||
await store.acquire(user_id, server_id, token, ttl=ttl)
|
||||
await store.save_record(batch_id, reservation, ttl=ttl)
|
||||
```
|
||||
- Keys and values already in Redis are a wire format. Old and new pods read each other's writes during a rolling deploy, so a move keeps them byte for byte, including `json.dumps` spacing and ASCII escaping, float formatting, and seconds versus milliseconds. Changing a key, codec or TTL unit is never part of a move. It is its own change that writes a new versioned key and reads both for one release
|
||||
- Response cache identity (normalization, hashing into a logical key) is backend independent and is not store logic. Reuse or move the Rust `cache_response::cache_key`, never write a third key builder
|
||||
|
||||
## Asked to move existing access behind a store
|
||||
|
||||
- Move the key builders, codec, TTLs, Lua and decoding verbatim. Keep policy in the caller
|
||||
- Keep the caller's outcome per failure. If it failed closed on any exception, it fails closed on any `RedisFailure`
|
||||
- Read how the replaced call failed before mapping it. `async_get_cache` and `async_set_cache` swallow Redis errors and return `None`, `async_delete_cache`, `async_increment` and registered scripts raise, and every method behind `_redis_circuit_breaker_guard` raises `RedisCircuitBreakerOpenError` when the breaker is open. So one `RedisFailure` can map to different outcomes by `kind`, and the caller matches on `kind` to keep each one
|
||||
- The one accepted difference is the breaker. A path that called the raw redis-py client skipped it and now goes through it. Say so in the PR
|
||||
- The move leaves no caller holding a key or a codec for that keyspace. Change the operations to take domain inputs in the same change, never as a follow-up
|
||||
- Delete the caller's own timing, service hooks and Redis health logging. The transport emits for it now
|
||||
- Move one keyspace per change. An extraction is not a Rust port
|
||||
- `RedisCache` is a connection object wearing a cache name. Its own cache verbs move onto the transport one method at a time, each conversion deleting that method's inline logging, until what is left is the connection, the breaker and a thin facade
|
||||
- Pipelined and batched access moves onto a transport batch, pub/sub onto `publish` / `subscribe`, and admin onto the admin methods. Vector and semantic search keep their own clients
|
||||
|
||||
## Observability
|
||||
|
||||
`ServiceLogging` (`litellm/_service_logger.py`) is a shared infra-health channel with many producers (Redis, Postgres, the spend queues, auth, router) and three sinks (`prometheus_system`, `datadog`, OTel). Redis work only decides where the Redis producer lives, never touches the sink side
|
||||
|
||||
- `RedisServiceEvents` (`litellm/redis/transport/events.py`, a frozen dataclass holding the `ServiceLogging`) is the only place REDIS service events for Python traffic are emitted. Its one method, `observe(call_type, call)`, times the call, logs a failure with the caller's service target, emits one success or failure event in the caller's task, and returns the result unchanged. The transport calls it with the method name as `call_type`. Stores and callers emit nothing
|
||||
- When no `service_callback` is configured, which is the default, `observe` runs the call and logs the failure without walking the stack or building an event
|
||||
- Rust never calls back into Python per Redis operation. A per-op host call costs a tokio task, a blocking thread, the GIL and an event loop per op, and buffering plus a flusher loses the OTel parent and needs a background task the SDK does not have
|
||||
- A native store holds the same `RedisServiceEvents` and runs each operation through `observe`, so native traffic reports from Python exactly like transport traffic, with the store operation name as `call_type`
|
||||
- The breaker for the native client is `RedisConfig::breaker` in Rust. A native store never holds a Python `RedisCircuitBreaker`, one breaker per socket. Breaker transitions are rare and may reach Python through one host call that folds the Rust breaker's state into `redis_breaker_state_counts()`
|
||||
- Keep metric names and label names as they are. Dashboards alert on `litellm_redis_*`. The one accepted label value change is `call_type`, which becomes the transport method name when a keyspace moves, and `prometheus_services.py` writes it into `function_name`
|
||||
|
||||
## Asked to port a store to Rust
|
||||
|
||||
Confirm with the user before the first port, since the bridge for it does not exist yet
|
||||
|
||||
- The keyspace's module in `litellm-rust/crates/redis-store` implements the same operations as async fns on `crates/redis-client`. Its `AGENTS.md` has the crate rules. Rust callers use the same module, so there is one implementation per operation
|
||||
- One `#[pyclass]` per store in `crates/python-bridge/src/redis/`, built on one shared native client handle, so Python opens no extra connection per store
|
||||
- A `Native<Keyspace>Store` in the Python module implements the same `Protocol` over the pyclass, each operation through `RedisServiceEvents.observe`, so callers do not change. The implementation is picked at construction, and the transport-backed one stays for config the native client does not support yet (Sentinel, cloud auth, TLS cert options)
|
||||
- Results come back as frozen Pydantic models generated from the Rust types, so callers keep matching on the same classes. A decode error maps to `RedisFailure(kind="decode")`
|
||||
- The codec writes the exact bytes Python writes. `json.dumps` escapes non-ASCII and `serde_json` does not, so a Rust pod could not release a lock a Python pod took
|
||||
- Each port ships cross-implementation tests against one Redis, Python writes and Rust reads and the reverse, covering keys, values, expiry and atomicity through a rolling deploy
|
||||
|
||||
## Tests
|
||||
|
||||
- Test a store through its caller against `fake_redis_cache(server)` from `tests/_support/fake_redis.py`, a real `RedisCache` on a `fakeredis.FakeServer`, so namespace, breaker and failure mapping are the production ones. No mocks of our own code
|
||||
- Cover each result the caller acts on, that the TTL lands (`pttl` on a raw `FakeAsyncRedis`), and an outage (`server.connected = False`)
|
||||
- When a keyspace moves, seed with the old writer and read with the new reader, and the reverse, so old and new pods interoperate
|
||||
- Assert returned values and Redis state, never the commands sent
|
||||
- Rust store tests run against real Redis from `litellm-redis-testing` on both topologies
|
||||
|
||||
## Decisions from the first slices
|
||||
|
||||
The first transport and store slices settled most of the shape. Keep what is listed under "Keep" as is, and rework anything that still has a "Change" item before extending it
|
||||
|
||||
Keep
|
||||
|
||||
- `RedisCache.transport()` builds the transport over the cache's own client, namespace and breaker, no second connection, no second breaker
|
||||
- `RedisFailure` as a value with kinds `transport`, `server`, `decode`, `breaker_open` and a `timeout` flag, and a failed write as an unknown outcome
|
||||
- Namespace applied once inside the transport, TTL from the caller as a `timedelta` sent in milliseconds, replies as undecoded bytes
|
||||
- The two steps around every redis-py command, the breaker and the events, one REDIS service event per command, none for `breaker_open`, one WARNING per failure through the timeout throttle with the caller's `with_service_target` in the line
|
||||
- `call_type` becoming the transport method name (`get`, `set`, `delete`) when a caller moves off `async_*_cache`
|
||||
- The store shape, a `Protocol` of domain operations plus a frozen dataclass over the transport that owns keys, codec, expiry, Lua and decoding, with policy in the caller
|
||||
- Byte for byte wire compatibility on a move, including the response polling double `json.dumps`, proven by seeding with the old writer and reading with the old reader
|
||||
- Tests through the caller on `fake_redis_cache(server)`, asserting values and Redis state, never commands sent
|
||||
- The Rust store crate shape, async fns on `crates/redis-client` returning domain types, one pyclass per store, a `Native<Keyspace>Store` behind the same `Protocol` picked at construction, frozen Pydantic results, cross-implementation tests on one Redis
|
||||
|
||||
Change
|
||||
|
||||
- Command dataclasses (`Get`, `Set`, `Delete`) run through an overloaded `execute` with a `match` per backend. With redis-py as the only backend they only add a second place to edit per command. The transport is plain methods
|
||||
|
||||
```python
|
||||
# before
|
||||
await transport.execute(Set(key, value, ttl=ttl, nx=True))
|
||||
await transport.execute(Get(key))
|
||||
|
||||
# after
|
||||
await transport.set(key, value, ttl=ttl, nx=True)
|
||||
await transport.get(key)
|
||||
```
|
||||
|
||||
- A pyo3 `NativeRedisTransport` over the Rust client behind the same Protocol. It mixes the stacks and would have to grow toward redis-py's command set. The Rust path is store level only
|
||||
- "Native stores skip the Python breaker and events, so both have to come from Rust first". Half right. The breaker comes from Rust, the events do not
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class NativeLockStore:
|
||||
native: NativeLockStoreHandle # pyclass on the shared Rust client, breaker inside
|
||||
events: RedisServiceEvents # no Python breaker here, one breaker per socket
|
||||
|
||||
async def acquire(self, key: str, owner: str, *, ttl: timedelta) -> bool | RedisFailure:
|
||||
return await self.events.observe("acquire", lambda: _reply(self.native.acquire(key, owner, ttl_ms(ttl))))
|
||||
```
|
||||
|
||||
- A Rust `RedisEvents` implementation that reports every native call to `ServiceLogging` through a host call. Delete the per-call path and keep only the breaker transition
|
||||
- A transport framed as "the backend swaps to the Rust client without touching callers". The transport is the redis-py seam, nothing else goes behind it
|
||||
- The events step walking the stack for `caller` and spawning a task per command whether or not a sink is configured. Check `ServiceLogging.has_sinks()` first
|
||||
- The names `Guard`, `BreakerGuard` and `ReportingGuard`, inherited from the `_redis_circuit_breaker_guard` decorator. Only the breaker guards anything. The breaker step is a function, `run_under_breaker(breaker, name, admit, call)`, which wraps `_run_under_circuit_breaker` and maps its exceptions to `RedisFailure`, and the events step is `RedisServiceEvents.observe`. The transport holds the breaker, the cache's `_admit_async_call` and the events as fields and there is no generic wrapper Protocol between them. Reimplementing the breaker admission in the transport, with `allow()` and a bare `record_failure()`, drops the generation check and the timeout class and is not an option
|
||||
- Lua through `async_register_script`, a per-loop executor cached in `in_memory_llm_clients_cache`. `transport.eval` takes the script text and does `EVALSHA` with `EVAL` fallback, no registration and no per-loop state
|
||||
- Lists in replies. `mget` returns a tuple with one slot per key and `eval` returns `int | bytes | None | tuple[...]`, a Lua `false` as `None`
|
||||
|
||||
## Scope
|
||||
|
||||
- Python moves ship first. Every raw redis-py keyspace outside `litellm/caching/` goes behind a store on the transport before anything under `litellm-rust/` changes for Redis. The first Rust port is the pod lock keyspace, since it is low traffic and its owner-checked release proves both stacks write the same `json.dumps` encoded owner
|
||||
- A keyspace moves whole in one commit, every operation the caller runs on those keys. Moving the read and leaving the write on `RedisCache` is not a move
|
||||
- Do only what was asked. An extraction is not a store move, a store move is not a Rust port, and a Rust port of one keyspace does not switch another
|
||||
- A Rust store that already exists does not by itself justify switching its Python caller. Switching callers is its own change with live verification
|
||||
1
.github/workflows/test-unit.yml
vendored
1
.github/workflows/test-unit.yml
vendored
|
|
@ -669,6 +669,7 @@ jobs:
|
|||
- shard: caching-local
|
||||
test-path: >-
|
||||
tests/unit/caching
|
||||
tests/unit/redis
|
||||
python-version: "3.12"
|
||||
workers: 2
|
||||
reruns: 0
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
import asyncio
|
||||
from collections.abc import Callable, Coroutine
|
||||
from collections.abc import Callable, Coroutine, Mapping
|
||||
from datetime import datetime, timedelta
|
||||
from typing import TYPE_CHECKING, Any, Final, Protocol
|
||||
|
||||
|
|
@ -111,6 +111,9 @@ class ServiceLogging(CustomLogger):
|
|||
return _as_otel_logger(open_telemetry_logger)
|
||||
return None
|
||||
|
||||
def has_sinks(self) -> bool:
|
||||
return bool(litellm.service_callback)
|
||||
|
||||
@staticmethod
|
||||
def _sync_dispatch_loop() -> asyncio.AbstractEventLoop | None:
|
||||
"""The event loop a blocking caller can dispatch on, or ``None`` if it has none."""
|
||||
|
|
@ -121,7 +124,7 @@ class ServiceLogging(CustomLogger):
|
|||
return None if loop.is_closed() else loop
|
||||
|
||||
@staticmethod
|
||||
async def _emit_guarded(hook: Callable[[], Coroutine[object, object, None]]) -> None:
|
||||
async def emit_guarded(hook: Callable[[], Coroutine[object, object, None]]) -> None:
|
||||
"""Emit one service event, absorbing anything the callbacks raise.
|
||||
|
||||
Monitoring must not break the call it monitors. Sync callers are the ones that
|
||||
|
|
@ -144,11 +147,11 @@ class ServiceLogging(CustomLogger):
|
|||
loop: Final = ServiceLogging._sync_dispatch_loop()
|
||||
try:
|
||||
if loop is None:
|
||||
asyncio.run(ServiceLogging._emit_guarded(hook))
|
||||
asyncio.run(ServiceLogging.emit_guarded(hook))
|
||||
elif loop.is_running():
|
||||
loop.create_task(ServiceLogging._emit_guarded(hook))
|
||||
loop.create_task(ServiceLogging.emit_guarded(hook))
|
||||
else:
|
||||
loop.run_until_complete(ServiceLogging._emit_guarded(hook))
|
||||
loop.run_until_complete(ServiceLogging.emit_guarded(hook))
|
||||
except Exception as e:
|
||||
verbose_logger.exception("Error dispatching service event - %s", e)
|
||||
|
||||
|
|
@ -224,15 +227,16 @@ class ServiceLogging(CustomLogger):
|
|||
parent_otel_span: Span | None = None,
|
||||
start_time: datetime | float | None = None,
|
||||
end_time: datetime | float | None = None,
|
||||
event_metadata: dict | None = None,
|
||||
event_metadata: Mapping[str, object] | None = None,
|
||||
caller: str | None = None,
|
||||
):
|
||||
) -> None:
|
||||
"""
|
||||
- For counting if the redis, postgres call is successful
|
||||
"""
|
||||
if self.mock_testing:
|
||||
self.mock_testing_async_success_hook += 1
|
||||
|
||||
metadata: Final = None if event_metadata is None else dict(event_metadata)
|
||||
payload: Final = ServiceLoggerPayload(
|
||||
is_error=False,
|
||||
error=None,
|
||||
|
|
@ -241,7 +245,7 @@ class ServiceLogging(CustomLogger):
|
|||
call_type=call_type,
|
||||
caller=caller,
|
||||
target=current_service_target() if service == ServiceTypes.REDIS else None,
|
||||
event_metadata=event_metadata,
|
||||
event_metadata=metadata,
|
||||
)
|
||||
|
||||
# OTel loggers already fired this event. ``service_callback`` can hold more
|
||||
|
|
@ -262,7 +266,7 @@ class ServiceLogging(CustomLogger):
|
|||
parent_otel_span=parent_otel_span,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
event_metadata=event_metadata,
|
||||
event_metadata=metadata,
|
||||
)
|
||||
else:
|
||||
_otel_logger_to_use = self._resolve_otel_service_logger(callback)
|
||||
|
|
@ -278,7 +282,7 @@ class ServiceLogging(CustomLogger):
|
|||
parent_otel_span=parent_otel_span,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
event_metadata=event_metadata,
|
||||
event_metadata=metadata,
|
||||
)
|
||||
|
||||
async def init_prometheus_services_logger_if_none(self):
|
||||
|
|
@ -325,9 +329,9 @@ class ServiceLogging(CustomLogger):
|
|||
parent_otel_span: Span | None = None,
|
||||
start_time: datetime | float | None = None,
|
||||
end_time: float | datetime | None = None,
|
||||
event_metadata: dict | None = None,
|
||||
event_metadata: Mapping[str, object] | None = None,
|
||||
caller: str | None = None,
|
||||
):
|
||||
) -> None:
|
||||
"""
|
||||
- For counting if the redis, postgres call is unsuccessful
|
||||
"""
|
||||
|
|
@ -340,6 +344,7 @@ class ServiceLogging(CustomLogger):
|
|||
elif isinstance(error, str):
|
||||
error_message = error
|
||||
|
||||
metadata: Final = None if event_metadata is None else dict(event_metadata)
|
||||
payload: Final = ServiceLoggerPayload(
|
||||
is_error=True,
|
||||
error=error_message,
|
||||
|
|
@ -348,7 +353,7 @@ class ServiceLogging(CustomLogger):
|
|||
call_type=call_type,
|
||||
caller=caller,
|
||||
target=current_service_target() if service == ServiceTypes.REDIS else None,
|
||||
event_metadata=event_metadata,
|
||||
event_metadata=metadata,
|
||||
)
|
||||
|
||||
# Dedupe OTel loggers per event — see ``async_service_success_hook`` for why
|
||||
|
|
@ -369,7 +374,7 @@ class ServiceLogging(CustomLogger):
|
|||
parent_otel_span=parent_otel_span,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
event_metadata=event_metadata,
|
||||
event_metadata=metadata,
|
||||
)
|
||||
else:
|
||||
_otel_logger_to_use = self._resolve_otel_service_logger(callback)
|
||||
|
|
@ -387,7 +392,7 @@ class ServiceLogging(CustomLogger):
|
|||
parent_otel_span=parent_otel_span,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
event_metadata=event_metadata,
|
||||
event_metadata=metadata,
|
||||
)
|
||||
|
||||
async def async_post_call_failure_hook(
|
||||
|
|
|
|||
|
|
@ -1,10 +1,10 @@
|
|||
"""One Redis pipeline for several independent operations, each with its own result and its own failure.
|
||||
"""``RedisCache``-shaped operations on the transport's pipeline engine, and the request scope that hands each
|
||||
backend one engine per request.
|
||||
|
||||
A ``RedisBatch`` collects MGETs, Lua scripts and increments declared by unrelated callers and sends them
|
||||
in one ``pipeline(transaction=False)`` round trip. Every declaration returns an awaitable; awaiting one
|
||||
flushes whatever has been declared so far, so callers keep their existing ``await`` shape and their own
|
||||
error handling while sharing the wire. Redis Cluster clients run each operation on its own, as before:
|
||||
a cluster pipeline is per node anyway and the existing per-operation paths already group by slot.
|
||||
A ``RedisBatch`` collects MGETs, Lua scripts and increments declared by unrelated callers and sends them in one
|
||||
round trip, together with any transport commands declared on the same engine. Every declaration returns an
|
||||
awaitable; awaiting one flushes whatever has been declared so far, so callers keep their existing ``await`` shape
|
||||
and their own error handling while sharing the wire. Redis Cluster clients run each operation on its own.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -12,131 +12,58 @@ from __future__ import annotations
|
|||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
import weakref
|
||||
from collections.abc import Awaitable, Callable, Generator, Mapping, Sequence
|
||||
from collections.abc import Awaitable, Callable, Mapping, Sequence
|
||||
from contextvars import ContextVar, Token
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import timedelta
|
||||
from types import MappingProxyType, TracebackType
|
||||
from typing import Final, Generic, Protocol, TypeVar
|
||||
from typing import Final, Protocol
|
||||
|
||||
from litellm._internal_context import (
|
||||
REDIS_FAMILIES_METADATA_KEY,
|
||||
current_service_target,
|
||||
service_caller,
|
||||
service_target,
|
||||
)
|
||||
from litellm._logging import verbose_logger
|
||||
from litellm.caching.redis_cache import (
|
||||
RedisCache,
|
||||
_get_call_stack_info, # pyright: ignore[reportPrivateUsage] # same caller chain every RedisCache method reports
|
||||
_run_under_circuit_breaker, # pyright: ignore[reportPrivateUsage] # same health signal as every RedisCache method
|
||||
log_redis_failure,
|
||||
)
|
||||
from litellm.caching.redis_cache import RedisCache
|
||||
from litellm.caching.redis_cluster_cache import RedisClusterCache
|
||||
from litellm.types.services import ServiceTypes
|
||||
from litellm.redis.transport.batch import (
|
||||
MIXED_PIPELINE_TARGET,
|
||||
BatchResult,
|
||||
PipelineBackend,
|
||||
PipelineBatch,
|
||||
SettledHook,
|
||||
)
|
||||
from litellm.redis.transport.client import RedisPipeline
|
||||
from litellm.redis.transport.contract import Arg
|
||||
|
||||
__all__ = (
|
||||
"MIXED_PIPELINE_TARGET",
|
||||
"POST_CALL_FLUSH_DEADLINE_SECONDS",
|
||||
"BatchResult",
|
||||
"RedisBatch",
|
||||
"RegisteredScript",
|
||||
"RequestRedisBatches",
|
||||
"SettledHook",
|
||||
"active_post_call_redis_batch",
|
||||
"active_request_redis_batch",
|
||||
"active_request_redis_batches",
|
||||
"drain_post_call_redis_batches",
|
||||
"flush_post_call_redis_batches",
|
||||
"post_call_batch_engine",
|
||||
"request_batch_engine",
|
||||
"request_redis_batch_scope",
|
||||
)
|
||||
|
||||
_T = TypeVar("_T")
|
||||
_ScriptArg = str | bytes | int | float
|
||||
SettledHook = Callable[[asyncio.Future[_T]], Awaitable[None] | None]
|
||||
POST_CALL_FLUSH_DEADLINE_SECONDS: Final = 1.0
|
||||
|
||||
|
||||
class RegisteredScript(Protocol):
|
||||
def __call__(self, keys: Sequence[str], args: Sequence[_ScriptArg]) -> Awaitable[object]: ...
|
||||
def __call__(self, keys: Sequence[str], args: Sequence[Arg]) -> Awaitable[object]: ...
|
||||
|
||||
|
||||
class _RedisPipeline(Protocol):
|
||||
def mget(self, keys: Sequence[str]) -> object: ...
|
||||
def evalsha(self, sha: str, numkeys: int, *keys_and_args: _ScriptArg) -> object: ...
|
||||
def incrbyfloat(self, name: str, amount: float) -> object: ...
|
||||
def expire(self, name: str, time: timedelta) -> object: ...
|
||||
def set(self, name: str, value: str, ex: timedelta | None = None) -> object: ...
|
||||
def delete(self, *names: str) -> object: ...
|
||||
async def execute(self, raise_on_error: bool = True) -> list[object]: ...
|
||||
|
||||
|
||||
class _Op(Generic[_T]):
|
||||
"""One declared operation: how many pipeline replies it consumes, how to turn them into a result, and
|
||||
how to run on its own when the batch cannot pipeline (cluster client, or a reply the pipeline cannot
|
||||
settle, like NOSCRIPT)."""
|
||||
|
||||
__slots__ = ("caller", "future", "settled_hooks", "target")
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.future: Final[asyncio.Future[_T]] = asyncio.get_running_loop().create_future()
|
||||
self.future.add_done_callback(_mark_retrieved)
|
||||
self.settled_hooks: Final[list[SettledHook[_T]]] = [] # mutable-ok: append-only registry
|
||||
self.target: Final = current_service_target()
|
||||
self.caller: Final = _get_call_stack_info()
|
||||
|
||||
async def run_settled_hooks(self) -> None:
|
||||
for hook in self.settled_hooks:
|
||||
await self._run_settled_hook(hook)
|
||||
|
||||
async def _run_settled_hook(self, hook: SettledHook[_T]) -> None:
|
||||
try:
|
||||
follow_up: Final = hook(self.future)
|
||||
if follow_up is not None:
|
||||
await follow_up
|
||||
except Exception as e: # noqa: BLE001 # one owner's follow-up must not stop the others
|
||||
verbose_logger.warning("redis batch settled hook failed: %s", e)
|
||||
|
||||
def enqueue(self, pipe: _RedisPipeline) -> int:
|
||||
raise NotImplementedError
|
||||
|
||||
def resolve(self, replies: Sequence[object]) -> _T:
|
||||
raise NotImplementedError
|
||||
|
||||
async def run_alone(self) -> _T:
|
||||
raise NotImplementedError
|
||||
|
||||
def settle(self, replies: Sequence[object]) -> Awaitable[None] | None:
|
||||
"""Resolve from pipeline replies; return a coroutine when the op has to be retried on its own."""
|
||||
failure: Final = next((reply for reply in replies if isinstance(reply, Exception)), None)
|
||||
if failure is None:
|
||||
try:
|
||||
self.future.set_result(self.resolve(replies))
|
||||
except Exception as e: # noqa: BLE001 # a reply this op cannot decode fails this op alone
|
||||
self.future.set_exception(e)
|
||||
return None
|
||||
if _is_missing_script(failure):
|
||||
return self._settle_alone()
|
||||
self.future.set_exception(failure)
|
||||
return None
|
||||
|
||||
async def _settle_alone(self) -> None:
|
||||
try:
|
||||
with service_target(self.target), service_caller(self.caller):
|
||||
self.future.set_result(await self.run_alone())
|
||||
except Exception as e: # noqa: BLE001 # the declaring caller owns the failure of its own operation
|
||||
self.future.set_exception(e)
|
||||
|
||||
|
||||
def _is_missing_script(failure: Exception) -> bool:
|
||||
"""Imported lazily: this module is reachable from a base ``import litellm`` while redis is not a base dependency."""
|
||||
from redis.exceptions import NoScriptError
|
||||
|
||||
return isinstance(failure, NoScriptError)
|
||||
|
||||
|
||||
def _mark_retrieved(future: asyncio.Future[object]) -> None:
|
||||
"""A caller that stops awaiting (cancelled request) must not leave an 'exception never retrieved' log."""
|
||||
if not future.cancelled():
|
||||
future.exception()
|
||||
|
||||
|
||||
class _MGet(_Op[Mapping[str, object]]):
|
||||
class _MGet:
|
||||
__slots__ = ("_keys", "_redis_cache")
|
||||
|
||||
def __init__(self, redis_cache: RedisCache, keys: Sequence[str]) -> None:
|
||||
super().__init__()
|
||||
self._redis_cache: Final = redis_cache
|
||||
self._keys: Final[tuple[str, ...]] = tuple(dict.fromkeys(keys))
|
||||
|
||||
def enqueue(self, pipe: _RedisPipeline) -> int:
|
||||
def enqueue(self, pipe: RedisPipeline) -> int:
|
||||
pipe.mget(tuple(self._redis_cache.check_and_fix_namespace(key=key) for key in self._keys))
|
||||
return 1
|
||||
|
||||
|
|
@ -148,6 +75,9 @@ class _MGet(_Op[Mapping[str, object]]):
|
|||
{key: self._redis_cache._get_cache_logic(value) for key, value in zip(self._keys, values)} # pyright: ignore[reportPrivateUsage, reportUnknownMemberType, reportUnknownArgumentType] # shared decode with async_batch_get_cache
|
||||
)
|
||||
|
||||
def recover(self, error: Exception) -> Mapping[str, object]:
|
||||
raise error
|
||||
|
||||
async def run_alone(self) -> Mapping[str, object]:
|
||||
found: Mapping[str, object] = await self._redis_cache.async_batch_get_cache(key_list=list(self._keys)) # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] # untyped cache API
|
||||
if any(key not in found for key in self._keys):
|
||||
|
|
@ -155,7 +85,7 @@ class _MGet(_Op[Mapping[str, object]]):
|
|||
return found
|
||||
|
||||
|
||||
class _Script(_Op[object]):
|
||||
class _Script:
|
||||
__slots__ = ("_args", "_keys", "_redis_cache", "_run", "_sha")
|
||||
|
||||
def __init__(
|
||||
|
|
@ -164,16 +94,15 @@ class _Script(_Op[object]):
|
|||
source: str,
|
||||
run: RegisteredScript,
|
||||
keys: Sequence[str],
|
||||
args: Sequence[_ScriptArg],
|
||||
args: Sequence[Arg],
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self._redis_cache: Final = redis_cache
|
||||
self._sha: Final = hashlib.sha1(source.encode()).hexdigest() # noqa: S324 # EVALSHA identifies scripts by SHA-1
|
||||
self._run: Final = run
|
||||
self._keys: Final[tuple[str, ...]] = tuple(keys)
|
||||
self._args: Final[tuple[_ScriptArg, ...]] = tuple(args)
|
||||
self._args: Final[tuple[Arg, ...]] = tuple(args)
|
||||
|
||||
def enqueue(self, pipe: _RedisPipeline) -> int:
|
||||
def enqueue(self, pipe: RedisPipeline) -> int:
|
||||
namespaced: Final = tuple(self._redis_cache.check_and_fix_namespace(key=key) for key in self._keys)
|
||||
pipe.evalsha(self._sha, len(namespaced), *namespaced, *self._args)
|
||||
return 1
|
||||
|
|
@ -181,21 +110,23 @@ class _Script(_Op[object]):
|
|||
def resolve(self, replies: Sequence[object]) -> object:
|
||||
return replies[0]
|
||||
|
||||
def recover(self, error: Exception) -> object:
|
||||
raise error
|
||||
|
||||
async def run_alone(self) -> object:
|
||||
return await self._run(keys=self._keys, args=self._args)
|
||||
|
||||
|
||||
class _Increment(_Op[float]):
|
||||
class _Increment:
|
||||
__slots__ = ("_key", "_redis_cache", "_ttl", "_value")
|
||||
|
||||
def __init__(self, redis_cache: RedisCache, key: str, value: float, ttl: int | None) -> None:
|
||||
super().__init__()
|
||||
self._redis_cache: Final = redis_cache
|
||||
self._key: Final = key
|
||||
self._value: Final = value
|
||||
self._ttl: Final = ttl
|
||||
|
||||
def enqueue(self, pipe: _RedisPipeline) -> int:
|
||||
def enqueue(self, pipe: RedisPipeline) -> int:
|
||||
name: Final = self._redis_cache.check_and_fix_namespace(key=self._key)
|
||||
pipe.incrbyfloat(name, self._value)
|
||||
if self._ttl is None:
|
||||
|
|
@ -209,6 +140,9 @@ class _Increment(_Op[float]):
|
|||
raise TypeError(f"INCRBYFLOAT reply is not numeric: {type(reply).__name__}")
|
||||
return float(reply)
|
||||
|
||||
def recover(self, error: Exception) -> float:
|
||||
raise error
|
||||
|
||||
async def run_alone(self) -> float:
|
||||
value: object = await self._redis_cache.async_increment(key=self._key, value=self._value, ttl=self._ttl) # pyright: ignore[reportUnknownMemberType] # untyped cache API
|
||||
if not isinstance(value, (int, float)):
|
||||
|
|
@ -216,19 +150,18 @@ class _Increment(_Op[float]):
|
|||
return float(value)
|
||||
|
||||
|
||||
class _Set(_Op[None]):
|
||||
class _Set:
|
||||
"""SET with the cache's TTL rules, same encoding as ``async_set_cache_pipeline_with_ttls``."""
|
||||
|
||||
__slots__ = ("_key", "_redis_cache", "_ttl", "_value")
|
||||
|
||||
def __init__(self, redis_cache: RedisCache, key: str, value: object, ttl: float | None) -> None:
|
||||
super().__init__()
|
||||
self._redis_cache: Final = redis_cache
|
||||
self._key: Final = key
|
||||
self._value: Final = value
|
||||
self._ttl: Final = ttl
|
||||
|
||||
def enqueue(self, pipe: _RedisPipeline) -> int:
|
||||
def enqueue(self, pipe: RedisPipeline) -> int:
|
||||
ttl: Final = self._redis_cache.get_ttl(ttl=self._ttl) # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType] # untyped cache API
|
||||
pipe.set(
|
||||
self._redis_cache.check_and_fix_namespace(key=self._key),
|
||||
|
|
@ -240,195 +173,104 @@ class _Set(_Op[None]):
|
|||
def resolve(self, replies: Sequence[object]) -> None:
|
||||
return None
|
||||
|
||||
def recover(self, error: Exception) -> None:
|
||||
raise error
|
||||
|
||||
async def run_alone(self) -> None:
|
||||
await self._redis_cache.async_set_cache_pipeline_with_ttls(((self._key, self._value, self._ttl),))
|
||||
|
||||
|
||||
class _Delete(_Op[None]):
|
||||
class _Delete:
|
||||
"""DEL of one key, the pipelined twin of ``async_delete_cache``."""
|
||||
|
||||
__slots__ = ("_key", "_redis_cache")
|
||||
|
||||
def __init__(self, redis_cache: RedisCache, key: str) -> None:
|
||||
super().__init__()
|
||||
self._redis_cache: Final = redis_cache
|
||||
self._key: Final = key
|
||||
|
||||
def enqueue(self, pipe: _RedisPipeline) -> int:
|
||||
def enqueue(self, pipe: RedisPipeline) -> int:
|
||||
pipe.delete(self._redis_cache.check_and_fix_namespace(key=self._key))
|
||||
return 1
|
||||
|
||||
def resolve(self, replies: Sequence[object]) -> None:
|
||||
return None
|
||||
|
||||
def recover(self, error: Exception) -> None:
|
||||
raise error
|
||||
|
||||
async def run_alone(self) -> None:
|
||||
await self._redis_cache.async_delete_cache(self._key)
|
||||
|
||||
|
||||
class BatchResult(Generic[_T]):
|
||||
"""Awaitable handle for one declared operation; awaiting it flushes the batch it belongs to."""
|
||||
|
||||
__slots__ = ("_batch", "_op")
|
||||
|
||||
def __init__(self, batch: RedisBatch, op: _Op[_T]) -> None:
|
||||
self._batch: Final = batch
|
||||
self._op: Final = op
|
||||
|
||||
def __await__(self) -> Generator[object, None, _T]:
|
||||
return self._wait().__await__()
|
||||
|
||||
async def _wait(self) -> _T:
|
||||
if not self._op.future.done():
|
||||
await self._batch.flush()
|
||||
return self._op.future.result()
|
||||
|
||||
@property
|
||||
def done(self) -> bool:
|
||||
return self._op.future.done()
|
||||
|
||||
def on_settled(self, hook: SettledHook[_T]) -> None:
|
||||
"""For an owner that does not await: runs inside the flush once this operation has its result or
|
||||
failure (or was cancelled with the pipeline), so the flush completes with the follow-up done."""
|
||||
self._op.settled_hooks.append(hook)
|
||||
def pipeline_backend(redis_cache: RedisCache) -> PipelineBackend:
|
||||
return PipelineBackend(
|
||||
client=redis_cache.init_async_client,
|
||||
breaker=redis_cache._circuit_breaker, # pyright: ignore[reportPrivateUsage] # same breaker as the cache's own methods
|
||||
admit=redis_cache._admit_async_call, # pyright: ignore[reportPrivateUsage] # same connection gate as the cache's own methods
|
||||
service_logging=redis_cache.service_logger_obj,
|
||||
runs_alone=isinstance(redis_cache, RedisClusterCache),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class RedisBatch:
|
||||
"""Operations declared here go out in one pipeline the next time any of them is awaited or ``flush`` runs."""
|
||||
|
||||
redis_cache: RedisCache
|
||||
name: str = "redis_batch"
|
||||
_pending: list[_Op[object]] = field(default_factory=list) # mutable-ok: drained by flush
|
||||
_flush_hooks: list[Callable[[], None]] = field(default_factory=list) # mutable-ok: append-only registry
|
||||
_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
|
||||
_misses: set[str] = field(default_factory=set) # mutable-ok: keys an MGET of this request read as absent
|
||||
flushes: int = 0
|
||||
__slots__ = ("engine", "redis_cache")
|
||||
|
||||
def __init__(self, redis_cache: RedisCache, name: str = "redis_batch") -> None:
|
||||
self.redis_cache: Final = redis_cache
|
||||
self.engine: Final = PipelineBatch(pipeline_backend(redis_cache), name=name)
|
||||
|
||||
@property
|
||||
def name(self) -> str:
|
||||
return self.engine.name
|
||||
|
||||
@property
|
||||
def flushes(self) -> int:
|
||||
return self.engine.flushes
|
||||
|
||||
@property
|
||||
def pending(self) -> int:
|
||||
return self.engine.pending
|
||||
|
||||
def mget(self, keys: Sequence[str]) -> BatchResult[Mapping[str, object]]:
|
||||
op: Final = _MGet(self.redis_cache, keys)
|
||||
op.future.add_done_callback(self._note_misses)
|
||||
return self._declare(op)
|
||||
result: Final = self.engine.declare(_MGet(self.redis_cache, keys))
|
||||
result.on_settled(self._note_misses)
|
||||
return result
|
||||
|
||||
def _note_misses(self, future: asyncio.Future[Mapping[str, object]]) -> None:
|
||||
if future.cancelled() or future.exception() is not None:
|
||||
return
|
||||
self._misses.update(key for key, value in future.result().items() if value is None)
|
||||
self.engine.note_missing(key for key, value in future.result().items() if value is None)
|
||||
|
||||
def read_as_missing(self, key: str) -> bool:
|
||||
"""True when an MGET on this batch already found no value under ``key`` and nothing has set it since,
|
||||
so a per-key GET later in the same request can be answered without another round trip."""
|
||||
return key in self._misses
|
||||
return self.engine.read_as_missing(key)
|
||||
|
||||
def script(
|
||||
self, source: str, run: RegisteredScript, keys: Sequence[str], args: Sequence[_ScriptArg]
|
||||
self, source: str, run: RegisteredScript, keys: Sequence[str], args: Sequence[Arg]
|
||||
) -> BatchResult[object]:
|
||||
return self._declare(_Script(self.redis_cache, source, run, keys, args))
|
||||
return self.engine.declare(_Script(self.redis_cache, source, run, keys, args))
|
||||
|
||||
def increment(self, key: str, value: float, ttl: int | None = None) -> BatchResult[float]:
|
||||
return self._declare(_Increment(self.redis_cache, key, value, ttl))
|
||||
return self.engine.declare(_Increment(self.redis_cache, key, value, ttl))
|
||||
|
||||
def set(self, key: str, value: object, ttl: float | None = None) -> BatchResult[None]:
|
||||
self._misses.discard(key)
|
||||
return self._declare(_Set(self.redis_cache, key, value, ttl))
|
||||
self.engine.note_present(key)
|
||||
return self.engine.declare(_Set(self.redis_cache, key, value, ttl))
|
||||
|
||||
def delete(self, key: str) -> BatchResult[None]:
|
||||
self._misses.add(key)
|
||||
return self._declare(_Delete(self.redis_cache, key))
|
||||
self.engine.note_missing((key,))
|
||||
return self.engine.declare(_Delete(self.redis_cache, key))
|
||||
|
||||
def add_flush_hook(self, hook: Callable[[], None]) -> None:
|
||||
"""Called at the start of every flush so lazily bound readers can declare their keys into the same trip."""
|
||||
self._flush_hooks.append(hook)
|
||||
|
||||
@property
|
||||
def pending(self) -> int:
|
||||
return len(self._pending)
|
||||
|
||||
def _declare(self, op: _Op[_T]) -> BatchResult[_T]:
|
||||
self._pending.append(op) # pyright: ignore[reportArgumentType] # heterogeneous ops share the flush loop
|
||||
return BatchResult(self, op)
|
||||
self.engine.add_flush_hook(hook)
|
||||
|
||||
async def flush(self) -> None:
|
||||
async with self._lock:
|
||||
for hook in self._flush_hooks:
|
||||
hook()
|
||||
ops: Final = tuple(self._pending)
|
||||
self._pending.clear()
|
||||
if not ops:
|
||||
return
|
||||
self.flushes += 1
|
||||
try:
|
||||
if isinstance(self.redis_cache, RedisClusterCache):
|
||||
await asyncio.gather(*(op._settle_alone() for op in ops)) # pyright: ignore[reportPrivateUsage] # batch owns its ops
|
||||
else:
|
||||
await self._flush_pipeline(ops)
|
||||
finally:
|
||||
for op in ops:
|
||||
if not op.future.done():
|
||||
op.future.cancel()
|
||||
await asyncio.gather(*(op.run_settled_hooks() for op in ops))
|
||||
|
||||
async def _flush_pipeline(self, ops: Sequence[_Op[object]]) -> None:
|
||||
start_time: Final = time.time()
|
||||
target, metadata = _pipeline_service_event(ops)
|
||||
widths: list[int] = [] # mutable-ok: filled while enqueuing
|
||||
|
||||
async def run() -> list[object]:
|
||||
client: Final = self.redis_cache.init_async_client()
|
||||
async with client.pipeline(transaction=False) as pipe:
|
||||
widths.extend(op.enqueue(pipe) for op in ops)
|
||||
return await pipe.execute(raise_on_error=False)
|
||||
|
||||
try:
|
||||
replies: Final = await _run_under_circuit_breaker(self.redis_cache._circuit_breaker, self.name, run) # pyright: ignore[reportPrivateUsage] # same breaker as the cache's own methods
|
||||
except Exception as e: # noqa: BLE001 # each declaring caller applies its own Redis fallback
|
||||
log_redis_failure(verbose_logger, logging.WARNING, f"{self.name}: pipeline of {len(ops)} ops failed", e)
|
||||
with service_target(target):
|
||||
asyncio.create_task(
|
||||
self.redis_cache.service_logger_obj.async_service_failure_hook(
|
||||
service=ServiceTypes.REDIS,
|
||||
duration=time.time() - start_time,
|
||||
error=e,
|
||||
call_type=self.name,
|
||||
start_time=start_time,
|
||||
end_time=time.time(),
|
||||
event_metadata=metadata,
|
||||
)
|
||||
)
|
||||
for op in ops:
|
||||
op.future.set_exception(e)
|
||||
return
|
||||
with service_target(target):
|
||||
asyncio.create_task(
|
||||
self.redis_cache.service_logger_obj.async_service_success_hook(
|
||||
service=ServiceTypes.REDIS,
|
||||
duration=time.time() - start_time,
|
||||
call_type=self.name,
|
||||
start_time=start_time,
|
||||
end_time=time.time(),
|
||||
event_metadata=metadata,
|
||||
)
|
||||
)
|
||||
retries: list[Awaitable[None]] = [] # mutable-ok: collected while slicing replies
|
||||
offset = 0
|
||||
for op, width in zip(ops, widths):
|
||||
retry = op.settle(replies[offset : offset + width])
|
||||
offset += width
|
||||
if retry is not None:
|
||||
retries.append(retry)
|
||||
if retries:
|
||||
await asyncio.gather(*retries)
|
||||
|
||||
|
||||
MIXED_PIPELINE_TARGET: Final = "mixed"
|
||||
|
||||
|
||||
def _pipeline_service_event(ops: Sequence[_Op[object]]) -> tuple[str | None, dict[str, int | str]]:
|
||||
"""The target and metadata of one pipeline flush: the one key family every op was declared under, or
|
||||
``"mixed"`` plus the sorted families when owners of several families share the trip."""
|
||||
families: Final = sorted({op.target for op in ops if op.target is not None})
|
||||
if len(families) > 1:
|
||||
return MIXED_PIPELINE_TARGET, {"op_count": len(ops), REDIS_FAMILIES_METADATA_KEY: ",".join(families)}
|
||||
return next(iter(families), None), {"op_count": len(ops)}
|
||||
await self.engine.flush()
|
||||
|
||||
|
||||
def _backend_key(redis_cache: RedisCache) -> object:
|
||||
|
|
@ -576,3 +418,14 @@ class request_redis_batch_scope:
|
|||
) -> None:
|
||||
if self._token is not None:
|
||||
_active_request_batches.reset(self._token)
|
||||
|
||||
|
||||
def request_batch_engine(redis_cache: RedisCache) -> PipelineBatch | None:
|
||||
"""The engine the transport declares request-scoped commands on, shared with this backend's ``RedisBatch``."""
|
||||
batch: Final = active_request_redis_batch(redis_cache)
|
||||
return None if batch is None else batch.engine
|
||||
|
||||
|
||||
def post_call_batch_engine(redis_cache: RedisCache) -> PipelineBatch | None:
|
||||
batch: Final = active_post_call_redis_batch(redis_cache)
|
||||
return None if batch is None else batch.engine
|
||||
|
|
|
|||
|
|
@ -42,6 +42,8 @@ from litellm.constants import (
|
|||
)
|
||||
from litellm.litellm_core_utils.core_helpers import get_parent_otel_span_from_kwargs
|
||||
from litellm.litellm_core_utils.coroutine_checker import coroutine_checker
|
||||
from litellm.redis.transport.client import AsyncPubSubClient, AsyncRedisCommands
|
||||
from litellm.redis.transport.contract import RedisFailure, RedisTransport
|
||||
from litellm.types.caching import (
|
||||
RedisPipelineIncrementOperation,
|
||||
RedisPipelineLpopOperation,
|
||||
|
|
@ -72,7 +74,7 @@ else:
|
|||
Span = Any
|
||||
|
||||
|
||||
class _AsyncRedisCommands(Protocol):
|
||||
class _AsyncRedisCommands(AsyncRedisCommands, Protocol):
|
||||
"""Async redis commands this cache issues.
|
||||
|
||||
redis-py's type stubs omit these methods on RedisCluster, so the union returned by
|
||||
|
|
@ -89,12 +91,8 @@ class _AsyncRedisCommands(Protocol):
|
|||
|
||||
def rpush(self, name: str, *values: str | bytes | float) -> Awaitable[int]: ...
|
||||
|
||||
def lpop(self, name: str, count: int | None = None) -> Awaitable[object]: ...
|
||||
|
||||
def pipeline(self, transaction: bool = True) -> "Pipeline[bytes]": ...
|
||||
|
||||
def eval(self, script: str, numkeys: int, *keys_and_args: str | bytes | float) -> Awaitable[object]: ...
|
||||
|
||||
|
||||
_GENERIC_CALLER_MODULES: Final = frozenset(
|
||||
{
|
||||
|
|
@ -151,7 +149,11 @@ def _decoded_counts(values: Sequence[bytes | str | None]) -> tuple[int | None, .
|
|||
|
||||
def _is_generic_caller_frame(frame: FrameType) -> bool:
|
||||
module: Final = frame.f_globals.get("__name__")
|
||||
return module in _GENERIC_CALLER_MODULES or frame.f_code.co_name in _GENERIC_CALLER_FRAME_NAMES
|
||||
return (
|
||||
module in _GENERIC_CALLER_MODULES
|
||||
or (isinstance(module, str) and module.startswith("litellm.redis."))
|
||||
or frame.f_code.co_name in _GENERIC_CALLER_FRAME_NAMES
|
||||
)
|
||||
|
||||
|
||||
def _ends_call_stack(frame: FrameType) -> bool:
|
||||
|
|
@ -512,25 +514,45 @@ def log_redis_failure(
|
|||
if isinstance(exc, RedisCircuitBreakerOpenError):
|
||||
logger.debug("%s: %s", message, exc, stacklevel=2)
|
||||
return
|
||||
exc_info: Final = exc if with_traceback else None
|
||||
if not is_redis_timeout_failure(exc):
|
||||
logger.log(level, "%s: %s", message, exc, exc_info=exc_info, stacklevel=2)
|
||||
_log_throttling_timeouts(
|
||||
logger, level, message, exc, timeout=is_redis_timeout_failure(exc), exc_info=exc if with_traceback else None
|
||||
)
|
||||
|
||||
|
||||
def log_redis_failure_value(logger: logging.Logger, level: int, message: str, failure: RedisFailure) -> None:
|
||||
if failure.kind == "breaker_open":
|
||||
logger.debug("%s: %s", message, failure.message, stacklevel=2)
|
||||
return
|
||||
_log_throttling_timeouts(logger, level, message, failure.message, timeout=failure.timeout, exc_info=None)
|
||||
|
||||
|
||||
def _log_throttling_timeouts(
|
||||
logger: logging.Logger,
|
||||
level: int,
|
||||
message: str,
|
||||
detail: object,
|
||||
*,
|
||||
timeout: bool,
|
||||
exc_info: BaseException | None,
|
||||
) -> None:
|
||||
if not timeout:
|
||||
logger.log(level, "%s: %s", message, detail, exc_info=exc_info, stacklevel=3)
|
||||
return
|
||||
suppressed: Final = _redis_timeout_log_throttle.admit()
|
||||
if suppressed is None:
|
||||
logger.debug("%s: %s", message, exc, stacklevel=2)
|
||||
logger.debug("%s: %s", message, detail, stacklevel=3)
|
||||
return
|
||||
if suppressed == 0:
|
||||
logger.log(level, "%s: %s", message, exc, exc_info=exc_info, stacklevel=2)
|
||||
logger.log(level, "%s: %s", message, detail, exc_info=exc_info, stacklevel=3)
|
||||
return
|
||||
logger.log(
|
||||
level,
|
||||
"%s: %s (%d more Redis timeouts since the previous Redis timeout line were logged at DEBUG)",
|
||||
message,
|
||||
exc,
|
||||
detail,
|
||||
suppressed,
|
||||
exc_info=exc_info,
|
||||
stacklevel=2,
|
||||
stacklevel=3,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -584,6 +606,12 @@ async def _run_under_circuit_breaker(
|
|||
return result
|
||||
|
||||
|
||||
def is_redis_error(exc: BaseException) -> bool:
|
||||
from redis.exceptions import RedisError
|
||||
|
||||
return isinstance(exc, RedisError)
|
||||
|
||||
|
||||
def _run_under_circuit_breaker_sync(
|
||||
breaker: RedisCircuitBreaker,
|
||||
name: str,
|
||||
|
|
@ -829,6 +857,27 @@ class RedisCache(BaseCache):
|
|||
def _async_commands(self) -> _AsyncRedisCommands:
|
||||
return self.init_async_client()
|
||||
|
||||
def _pubsub_commands(self) -> AsyncPubSubClient:
|
||||
return self.init_pubsub_client() # pyright: ignore[reportUnknownVariableType] # redis generics
|
||||
|
||||
def transport(self) -> RedisTransport:
|
||||
from litellm.caching.redis_batch import post_call_batch_engine, request_batch_engine
|
||||
from litellm.redis.transport.events import RedisServiceEvents
|
||||
from litellm.redis.transport.redis_py import RedisPyTransport
|
||||
|
||||
return RedisPyTransport(
|
||||
client=self._async_commands,
|
||||
namespace=self.namespace,
|
||||
breaker=self._circuit_breaker,
|
||||
admit=self._admit_async_call,
|
||||
events=RedisServiceEvents(self.service_logger_obj),
|
||||
multi_get=lambda keys: self._async_run_redis_mget_operation(list(keys)),
|
||||
server_version=self._parse_redis_major_version,
|
||||
pubsub_client=self._pubsub_commands,
|
||||
request_engine=lambda: request_batch_engine(self),
|
||||
post_call_engine=lambda: post_call_batch_engine(self),
|
||||
)
|
||||
|
||||
def check_and_fix_namespace(self, key: str) -> str:
|
||||
"""
|
||||
Make sure each key starts with the given namespace
|
||||
|
|
|
|||
13
litellm/redis/AGENTS.md
Normal file
13
litellm/redis/AGENTS.md
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
For adding Redis access or moving raw Redis access here, follow [.agents/skills/redis-access/SKILL.md](../../.agents/skills/redis-access/SKILL.md)
|
||||
|
||||
- `transport/contract.py` is the Python seam to redis-py. `RedisTransport` is plain methods covering every Redis use in Python, data commands, batches, pub/sub and admin, command shaped and never redis-py shaped. It is not a seam for the Rust client, native stores sit behind a store `Protocol` instead
|
||||
- `transport/client.py` holds the redis-py client shapes the transport drives, as Protocols with no redis import. `transport/batch.py` is the one pipeline engine, shared by transport batches and `RedisBatch`
|
||||
- Rules shared with `crates/redis-client`, so Python and Rust pods agree on one Redis
|
||||
- The namespace is applied here, once, including Lua `KEYS` and pub/sub channels. A key that already starts with `{namespace}:` is left alone
|
||||
- Every write takes its TTL from the caller, `None` means no expiry, there is no default
|
||||
- Replies are undecoded bytes, `mget` and `eval` arrays are tuples
|
||||
- Failures are values. `transport` covers timeouts and connection loss, where the command may still have applied. `server` means Redis answered with an error and the connection is fine
|
||||
- `transport/redis_py.py` is the only module here that imports redis-py. `transport/breaker.py` reuses `RedisCache`'s breaker admission and health classification, it never reimplements them
|
||||
- `transport/events.py` is the only place REDIS service events for transport traffic are emitted. Stores and callers emit nothing and do not log Redis health
|
||||
- `stores/` holds one module per keyspace, and a keyspace that has a store is only reached through it. A store is a `Protocol` of domain operations plus a frozen dataclass over `RedisTransport`. It owns its keys, value codec, expiry, Lua and decoding, and callers never see a Redis command
|
||||
- Keys, value bytes and TTL units in Redis are a wire format shared with older pods and Rust. Never change them in place
|
||||
0
litellm/redis/__init__.py
Normal file
0
litellm/redis/__init__.py
Normal file
0
litellm/redis/transport/__init__.py
Normal file
0
litellm/redis/transport/__init__.py
Normal file
255
litellm/redis/transport/batch.py
Normal file
255
litellm/redis/transport/batch.py
Normal file
|
|
@ -0,0 +1,255 @@
|
|||
import asyncio
|
||||
import time
|
||||
from collections.abc import Awaitable, Callable, Generator, Iterable, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Final, Generic, Protocol, TypeVar
|
||||
|
||||
from litellm._internal_context import (
|
||||
REDIS_FAMILIES_METADATA_KEY,
|
||||
current_service_target,
|
||||
service_caller,
|
||||
service_target,
|
||||
)
|
||||
from litellm._logging import verbose_logger
|
||||
from litellm.caching.redis_cache import (
|
||||
RedisCircuitBreaker,
|
||||
_get_call_stack_info, # pyright: ignore[reportPrivateUsage] # same caller chain every RedisCache method reports
|
||||
_run_under_circuit_breaker, # pyright: ignore[reportPrivateUsage] # same health signal as every RedisCache method
|
||||
)
|
||||
from litellm.redis.transport.client import PipelineClient, RedisPipeline
|
||||
from litellm.redis.transport.contract import Admit
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from litellm._service_logger import ServiceLogging
|
||||
|
||||
_T = TypeVar("_T")
|
||||
_T_co = TypeVar("_T_co", covariant=True)
|
||||
SettledHook = Callable[[asyncio.Future[_T]], Awaitable[None] | None]
|
||||
MIXED_PIPELINE_TARGET: Final = "mixed"
|
||||
|
||||
|
||||
class OpSpec(Protocol[_T_co]):
|
||||
"""How one operation rides a pipeline: the commands it queues, how its replies become its result, what a
|
||||
failed pipeline or command makes of it (raise to fail the op, return to settle it), and how it runs alone."""
|
||||
|
||||
def enqueue(self, pipe: RedisPipeline) -> int: ...
|
||||
|
||||
def resolve(self, replies: Sequence[object]) -> _T_co: ...
|
||||
|
||||
def recover(self, error: Exception) -> _T_co: ...
|
||||
|
||||
def run_alone(self) -> Awaitable[_T_co]: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PipelineBackend:
|
||||
client: Callable[[], PipelineClient]
|
||||
breaker: RedisCircuitBreaker
|
||||
admit: Admit
|
||||
service_logging: "ServiceLogging"
|
||||
runs_alone: bool
|
||||
|
||||
|
||||
def _mark_retrieved(future: asyncio.Future[object]) -> None:
|
||||
if not future.cancelled():
|
||||
future.exception()
|
||||
|
||||
|
||||
def _is_missing_script(failure: Exception) -> bool:
|
||||
from litellm.redis.transport.redis_py import is_missing_script
|
||||
|
||||
return is_missing_script(failure)
|
||||
|
||||
|
||||
class _Op(Generic[_T]):
|
||||
__slots__ = ("caller", "future", "settled_hooks", "spec", "target")
|
||||
|
||||
def __init__(self, spec: OpSpec[_T]) -> None:
|
||||
self.spec: Final = spec
|
||||
self.future: Final[asyncio.Future[_T]] = asyncio.get_running_loop().create_future()
|
||||
self.future.add_done_callback(_mark_retrieved)
|
||||
self.settled_hooks: Final[list[SettledHook[_T]]] = [] # mutable-ok: append-only registry
|
||||
self.target: Final = current_service_target()
|
||||
self.caller: Final = _get_call_stack_info()
|
||||
|
||||
async def run_settled_hooks(self) -> None:
|
||||
for hook in self.settled_hooks:
|
||||
await self._run_settled_hook(hook)
|
||||
|
||||
async def _run_settled_hook(self, hook: SettledHook[_T]) -> None:
|
||||
try:
|
||||
follow_up: Final = hook(self.future)
|
||||
if follow_up is not None:
|
||||
await follow_up
|
||||
except Exception as e: # noqa: BLE001 # one owner's follow-up must not stop the others
|
||||
verbose_logger.warning("redis batch settled hook failed: %s", e)
|
||||
|
||||
def settle(self, replies: Sequence[object]) -> Awaitable[None] | None:
|
||||
failure: Final = next((reply for reply in replies if isinstance(reply, Exception)), None)
|
||||
if failure is None:
|
||||
try:
|
||||
self.future.set_result(self.spec.resolve(replies))
|
||||
except Exception as e: # noqa: BLE001 # a reply this op cannot decode fails this op alone
|
||||
self.future.set_exception(e)
|
||||
return None
|
||||
if _is_missing_script(failure):
|
||||
return self.settle_alone()
|
||||
self.recover(failure)
|
||||
return None
|
||||
|
||||
def recover(self, error: Exception) -> None:
|
||||
try:
|
||||
self.future.set_result(self.spec.recover(error))
|
||||
except Exception as e: # noqa: BLE001 # the declaring caller owns the failure of its own operation
|
||||
self.future.set_exception(e)
|
||||
|
||||
async def settle_alone(self) -> None:
|
||||
try:
|
||||
with service_target(self.target), service_caller(self.caller):
|
||||
self.future.set_result(await self.spec.run_alone())
|
||||
except Exception as e: # noqa: BLE001 # the declaring caller owns the failure of its own operation
|
||||
self.future.set_exception(e)
|
||||
|
||||
|
||||
class BatchResult(Generic[_T]):
|
||||
"""Awaitable handle for one declared operation; awaiting it flushes the batch it belongs to."""
|
||||
|
||||
__slots__ = ("_batch", "_op")
|
||||
|
||||
def __init__(self, batch: "PipelineBatch", op: _Op[_T]) -> None:
|
||||
self._batch: Final = batch
|
||||
self._op: Final = op
|
||||
|
||||
def __await__(self) -> Generator[object, None, _T]:
|
||||
return self._wait().__await__()
|
||||
|
||||
async def _wait(self) -> _T:
|
||||
if not self._op.future.done():
|
||||
await self._batch.flush()
|
||||
return self._op.future.result()
|
||||
|
||||
@property
|
||||
def done(self) -> bool:
|
||||
return self._op.future.done()
|
||||
|
||||
def on_settled(self, hook: SettledHook[_T]) -> None:
|
||||
"""For an owner that does not await: runs inside the flush once this operation has its result or
|
||||
failure (or was cancelled with the pipeline), so the flush completes with the follow-up done."""
|
||||
self._op.settled_hooks.append(hook)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class PipelineBatch:
|
||||
"""Operations declared here go out in one pipeline the next time any of them is awaited or ``flush`` runs.
|
||||
A backend that cannot pipeline them (a cluster client) runs each on its own, unless the batch is a
|
||||
transaction, which always goes out as one MULTI/EXEC."""
|
||||
|
||||
backend: PipelineBackend
|
||||
name: str = "redis_batch"
|
||||
transaction: bool = False
|
||||
_pending: list[_Op[object]] = field(default_factory=list) # mutable-ok: drained by flush
|
||||
_flush_hooks: list[Callable[[], None]] = field(default_factory=list) # mutable-ok: append-only registry
|
||||
_lock: asyncio.Lock = field(default_factory=asyncio.Lock)
|
||||
_misses: set[str] = field(default_factory=set) # mutable-ok: keys a read of this batch found absent
|
||||
flushes: int = 0
|
||||
|
||||
def declare(self, spec: OpSpec[_T]) -> BatchResult[_T]:
|
||||
op: Final = _Op(spec)
|
||||
self._pending.append(op) # pyright: ignore[reportArgumentType] # heterogeneous ops share the flush loop
|
||||
return BatchResult(self, op)
|
||||
|
||||
def note_missing(self, keys: Iterable[str]) -> None:
|
||||
self._misses.update(keys)
|
||||
|
||||
def note_present(self, key: str) -> None:
|
||||
self._misses.discard(key)
|
||||
|
||||
def read_as_missing(self, key: str) -> bool:
|
||||
"""True when a read on this batch already found no value under ``key`` and nothing has set it since,
|
||||
so a per-key GET later in the same request can be answered without another round trip."""
|
||||
return key in self._misses
|
||||
|
||||
def add_flush_hook(self, hook: Callable[[], None]) -> None:
|
||||
"""Called at the start of every flush so lazily bound readers can declare their keys into the same trip."""
|
||||
self._flush_hooks.append(hook)
|
||||
|
||||
@property
|
||||
def pending(self) -> int:
|
||||
return len(self._pending)
|
||||
|
||||
async def flush(self) -> None:
|
||||
async with self._lock:
|
||||
for hook in self._flush_hooks:
|
||||
hook()
|
||||
ops: Final = tuple(self._pending)
|
||||
self._pending.clear()
|
||||
if not ops:
|
||||
return
|
||||
self.flushes += 1
|
||||
try:
|
||||
if self.backend.runs_alone and not self.transaction:
|
||||
await asyncio.gather(*(op.settle_alone() for op in ops))
|
||||
else:
|
||||
await self._flush_pipeline(ops)
|
||||
finally:
|
||||
for op in ops:
|
||||
if not op.future.done():
|
||||
op.future.cancel()
|
||||
await asyncio.gather(*(op.run_settled_hooks() for op in ops))
|
||||
|
||||
async def _flush_pipeline(self, ops: Sequence[_Op[object]]) -> None:
|
||||
from litellm.redis.transport.events import report_pipeline
|
||||
|
||||
start_time: Final = time.time()
|
||||
target, metadata = _pipeline_service_event(ops)
|
||||
widths: list[int] = [] # mutable-ok: filled while enqueuing
|
||||
|
||||
async def run() -> list[object]:
|
||||
async with self.backend.client().pipeline(transaction=self.transaction) as pipe:
|
||||
widths.extend(op.spec.enqueue(pipe) for op in ops)
|
||||
return await pipe.execute(raise_on_error=False)
|
||||
|
||||
try:
|
||||
replies: Final = await _run_under_circuit_breaker(
|
||||
self.backend.breaker, self.name, lambda: self.backend.admit(run)
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 # each declaring caller applies its own Redis fallback
|
||||
report_pipeline(
|
||||
self.backend.service_logging,
|
||||
self.name,
|
||||
op_count=len(ops),
|
||||
target=target,
|
||||
metadata=metadata,
|
||||
start_time=start_time,
|
||||
error=e,
|
||||
)
|
||||
for op in ops:
|
||||
op.recover(e)
|
||||
return
|
||||
report_pipeline(
|
||||
self.backend.service_logging,
|
||||
self.name,
|
||||
op_count=len(ops),
|
||||
target=target,
|
||||
metadata=metadata,
|
||||
start_time=start_time,
|
||||
error=None,
|
||||
)
|
||||
retries: list[Awaitable[None]] = [] # mutable-ok: collected while slicing replies
|
||||
offset = 0
|
||||
for op, width in zip(ops, widths):
|
||||
retry = op.settle(replies[offset : offset + width])
|
||||
offset += width
|
||||
if retry is not None:
|
||||
retries.append(retry)
|
||||
if retries:
|
||||
await asyncio.gather(*retries)
|
||||
|
||||
|
||||
def _pipeline_service_event(ops: Sequence[_Op[object]]) -> tuple[str | None, dict[str, int | str]]:
|
||||
"""The target and metadata of one pipeline flush: the one key family every op was declared under, or
|
||||
``"mixed"`` plus the sorted families when owners of several families share the trip."""
|
||||
families: Final = sorted({op.target for op in ops if op.target is not None})
|
||||
if len(families) > 1:
|
||||
return MIXED_PIPELINE_TARGET, {"op_count": len(ops), REDIS_FAMILIES_METADATA_KEY: ",".join(families)}
|
||||
return next(iter(families), None), {"op_count": len(ops)}
|
||||
36
litellm/redis/transport/breaker.py
Normal file
36
litellm/redis/transport/breaker.py
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
from collections.abc import Awaitable, Callable
|
||||
from typing import Final, TypeVar
|
||||
|
||||
from litellm.caching.redis_cache import (
|
||||
RedisCircuitBreaker,
|
||||
RedisCircuitBreakerOpenError,
|
||||
_is_redis_health_failure, # pyright: ignore[reportPrivateUsage] # same health classification as RedisCache
|
||||
_run_under_circuit_breaker, # pyright: ignore[reportPrivateUsage] # same admission and health signal as RedisCache
|
||||
is_redis_error,
|
||||
is_redis_timeout_failure,
|
||||
)
|
||||
from litellm.redis.transport.contract import Admit, RedisFailure
|
||||
|
||||
_T: Final = TypeVar("_T")
|
||||
|
||||
|
||||
async def run_under_breaker(
|
||||
breaker: RedisCircuitBreaker, name: str, admit: Admit, call: Callable[[], Awaitable[_T]]
|
||||
) -> _T | RedisFailure:
|
||||
try:
|
||||
return await _run_under_circuit_breaker(breaker, name, lambda: admit(call))
|
||||
except Exception as e:
|
||||
failure: Final = failure_from(e)
|
||||
if failure is None:
|
||||
raise
|
||||
return failure
|
||||
|
||||
|
||||
def failure_from(exc: BaseException) -> RedisFailure | None:
|
||||
if isinstance(exc, RedisCircuitBreakerOpenError):
|
||||
return RedisFailure(kind="breaker_open", message=str(exc))
|
||||
if _is_redis_health_failure(exc):
|
||||
return RedisFailure(kind="transport", message=str(exc), timeout=is_redis_timeout_failure(exc))
|
||||
if is_redis_error(exc):
|
||||
return RedisFailure(kind="server", message=str(exc))
|
||||
return None
|
||||
90
litellm/redis/transport/client.py
Normal file
90
litellm/redis/transport/client.py
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
from collections.abc import AsyncIterator, Awaitable, Sequence
|
||||
from contextlib import AbstractAsyncContextManager
|
||||
from datetime import timedelta
|
||||
from typing import Protocol
|
||||
|
||||
from litellm.redis.transport.contract import Arg
|
||||
|
||||
|
||||
class RedisPipeline(Protocol):
|
||||
def get(self, name: str) -> object: ...
|
||||
def mget(self, keys: Sequence[str]) -> object: ...
|
||||
def set(
|
||||
self, name: str, value: Arg, ex: timedelta | None = None, px: int | None = None, nx: bool = False
|
||||
) -> object: ...
|
||||
def delete(self, *names: str) -> object: ...
|
||||
def incrby(self, name: str, amount: int) -> object: ...
|
||||
def incrbyfloat(self, name: str, amount: float) -> object: ...
|
||||
def expire(self, name: str, time: timedelta) -> object: ...
|
||||
def pexpire(self, name: str, time: int) -> object: ...
|
||||
def ttl(self, name: str) -> object: ...
|
||||
def rpush(self, name: str, *values: Arg) -> object: ...
|
||||
def lpop(self, name: str, count: int | None = None) -> object: ...
|
||||
def ltrim(self, name: str, start: int, end: int) -> object: ...
|
||||
def sadd(self, name: str, *values: Arg) -> object: ...
|
||||
def eval(self, script: str, numkeys: int, *keys_and_args: Arg) -> object: ...
|
||||
def evalsha(self, sha: str, numkeys: int, *keys_and_args: Arg) -> object: ...
|
||||
def publish(self, channel: str, message: str | bytes) -> object: ...
|
||||
async def execute(self, raise_on_error: bool = True) -> list[object]: ...
|
||||
|
||||
|
||||
class PipelineClient(Protocol):
|
||||
def pipeline(self, transaction: bool = True) -> AbstractAsyncContextManager[RedisPipeline]: ...
|
||||
|
||||
|
||||
class AsyncRedisCommands(Protocol):
|
||||
def get(self, name: str) -> Awaitable[object]: ...
|
||||
|
||||
def mget(self, keys: Sequence[str]) -> Awaitable[object]: ...
|
||||
|
||||
def set(self, name: str, value: Arg, *, nx: bool, px: int | None) -> Awaitable[object]: ...
|
||||
|
||||
def delete(self, *names: str) -> Awaitable[object]: ...
|
||||
|
||||
def incrby(self, name: str, amount: int) -> Awaitable[object]: ...
|
||||
|
||||
def incrbyfloat(self, name: str, amount: float) -> Awaitable[object]: ...
|
||||
|
||||
def pexpire(self, name: str, time: int) -> Awaitable[object]: ...
|
||||
|
||||
def ttl(self, name: str) -> Awaitable[object]: ...
|
||||
|
||||
def rpush(self, name: str, *values: Arg) -> Awaitable[object]: ...
|
||||
|
||||
def lpop(self, name: str, count: int | None = None) -> Awaitable[object]: ...
|
||||
|
||||
def ltrim(self, name: str, start: int, end: int) -> Awaitable[object]: ...
|
||||
|
||||
def sadd(self, name: str, *values: Arg) -> Awaitable[object]: ...
|
||||
|
||||
def scan_iter(self, *, match: str, count: int) -> AsyncIterator[object]: ...
|
||||
|
||||
def eval(self, script: str, numkeys: int, *keys_and_args: Arg) -> Awaitable[object]: ...
|
||||
|
||||
def evalsha(self, sha: str, numkeys: int, *keys_and_args: Arg) -> Awaitable[object]: ...
|
||||
|
||||
def pipeline(self, transaction: bool = True) -> AbstractAsyncContextManager[RedisPipeline]: ...
|
||||
|
||||
def ping(self) -> Awaitable[object]: ...
|
||||
|
||||
def info(self) -> Awaitable[object]: ...
|
||||
|
||||
def client_list(self) -> Awaitable[object]: ...
|
||||
|
||||
def flushall(self) -> Awaitable[object]: ...
|
||||
|
||||
|
||||
class AsyncPubSub(Protocol):
|
||||
def subscribe(self, *channels: str) -> Awaitable[object]: ...
|
||||
|
||||
def get_message(self, *, ignore_subscribe_messages: bool, timeout: float) -> Awaitable[object]: ...
|
||||
|
||||
def reset(self) -> Awaitable[object]: ...
|
||||
|
||||
|
||||
class AsyncPubSubClient(Protocol):
|
||||
def publish(self, channel: str, message: str | bytes) -> Awaitable[object]: ...
|
||||
|
||||
def pubsub(self) -> AsyncPubSub: ...
|
||||
|
||||
def pipeline(self, transaction: bool = True) -> AbstractAsyncContextManager[RedisPipeline]: ...
|
||||
130
litellm/redis/transport/contract.py
Normal file
130
litellm/redis/transport/contract.py
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
import math
|
||||
from collections.abc import Awaitable, Callable, Generator, Mapping, Sequence
|
||||
from dataclasses import dataclass
|
||||
from datetime import timedelta
|
||||
from typing import Final, Literal, Protocol, TypeAlias, TypeVar
|
||||
|
||||
Arg: TypeAlias = str | bytes | int | float
|
||||
EvalReply: TypeAlias = int | bytes | None | tuple["EvalReply", ...]
|
||||
InfoScalar: TypeAlias = str | int | float
|
||||
InfoValue: TypeAlias = InfoScalar | Mapping[str, InfoScalar]
|
||||
Popped: TypeAlias = bytes | tuple[bytes, ...] | None
|
||||
|
||||
_T: Final = TypeVar("_T")
|
||||
_T_co: Final = TypeVar("_T_co", covariant=True)
|
||||
|
||||
|
||||
class Admit(Protocol):
|
||||
def __call__(self, call: Callable[[], Awaitable[_T]], /) -> Awaitable[_T]: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RedisFailure:
|
||||
kind: Literal["transport", "server", "decode", "breaker_open"]
|
||||
message: str
|
||||
timeout: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PubSubMessage:
|
||||
channel: bytes
|
||||
data: bytes
|
||||
|
||||
|
||||
def ttl_ms(ttl: timedelta) -> int:
|
||||
return max(1, math.ceil(ttl / timedelta(milliseconds=1)))
|
||||
|
||||
|
||||
class BatchReply(Protocol[_T_co]):
|
||||
def __await__(self) -> Generator[object, None, _T_co]: ...
|
||||
|
||||
@property
|
||||
def done(self) -> bool: ...
|
||||
|
||||
|
||||
class RedisCommandBatch(Protocol):
|
||||
def get(self, key: str) -> BatchReply[bytes | None | RedisFailure]: ...
|
||||
|
||||
def mget(self, keys: Sequence[str]) -> BatchReply[tuple[bytes | None, ...] | RedisFailure]: ...
|
||||
|
||||
def set(
|
||||
self, key: str, value: Arg, *, ttl: timedelta | None, nx: bool = False
|
||||
) -> BatchReply[bool | RedisFailure]: ...
|
||||
|
||||
def delete(self, *keys: str) -> BatchReply[int | RedisFailure]: ...
|
||||
|
||||
def incrby(self, key: str, amount: int) -> BatchReply[int | RedisFailure]: ...
|
||||
|
||||
def incrbyfloat(self, key: str, amount: float) -> BatchReply[float | RedisFailure]: ...
|
||||
|
||||
def expire(self, key: str, ttl: timedelta) -> BatchReply[bool | RedisFailure]: ...
|
||||
|
||||
def ttl(self, key: str) -> BatchReply[int | RedisFailure]: ...
|
||||
|
||||
def rpush(self, key: str, *values: Arg) -> BatchReply[int | RedisFailure]: ...
|
||||
|
||||
def lpop(self, key: str, count: int | None = None) -> BatchReply[Popped | RedisFailure]: ...
|
||||
|
||||
def ltrim(self, key: str, start: int, stop: int) -> BatchReply[bool | RedisFailure]: ...
|
||||
|
||||
def sadd(self, key: str, *members: Arg) -> BatchReply[int | RedisFailure]: ...
|
||||
|
||||
def eval(self, script: str, keys: Sequence[str], args: Sequence[Arg]) -> BatchReply[EvalReply | RedisFailure]: ...
|
||||
|
||||
async def flush(self) -> None: ...
|
||||
|
||||
|
||||
class RedisSubscription(Protocol):
|
||||
async def get_message(self, *, timeout: float) -> PubSubMessage | None | RedisFailure: ...
|
||||
|
||||
async def close(self) -> None | RedisFailure: ...
|
||||
|
||||
|
||||
class RedisTransport(Protocol):
|
||||
async def get(self, key: str) -> bytes | None | RedisFailure: ...
|
||||
|
||||
async def mget(self, keys: Sequence[str]) -> tuple[bytes | None, ...] | RedisFailure: ...
|
||||
|
||||
async def set(self, key: str, value: Arg, *, ttl: timedelta | None, nx: bool = False) -> bool | RedisFailure: ...
|
||||
|
||||
async def delete(self, *keys: str) -> int | RedisFailure: ...
|
||||
|
||||
async def incrby(self, key: str, amount: int) -> int | RedisFailure: ...
|
||||
|
||||
async def incrbyfloat(self, key: str, amount: float) -> float | RedisFailure: ...
|
||||
|
||||
async def expire(self, key: str, ttl: timedelta) -> bool | RedisFailure: ...
|
||||
|
||||
async def ttl(self, key: str) -> int | RedisFailure: ...
|
||||
|
||||
async def rpush(self, key: str, *values: Arg) -> int | RedisFailure: ...
|
||||
|
||||
async def lpop(self, key: str, count: int | None = None) -> Popped | RedisFailure: ...
|
||||
|
||||
async def ltrim(self, key: str, start: int, stop: int) -> bool | RedisFailure: ...
|
||||
|
||||
async def sadd(self, key: str, *members: Arg) -> int | RedisFailure: ...
|
||||
|
||||
async def scan(self, prefix: str, *, limit: int) -> tuple[bytes, ...] | RedisFailure: ...
|
||||
|
||||
async def eval(self, script: str, keys: Sequence[str], args: Sequence[Arg]) -> EvalReply | RedisFailure: ...
|
||||
|
||||
def batch(self, *, transaction: bool = False) -> RedisCommandBatch: ...
|
||||
|
||||
def request_batch(self) -> RedisCommandBatch | None: ...
|
||||
|
||||
def post_call_batch(self) -> RedisCommandBatch | None: ...
|
||||
|
||||
async def publish(self, channel: str, message: str | bytes) -> int | RedisFailure: ...
|
||||
|
||||
async def publish_all(self, channel: str, messages: Sequence[str | bytes]) -> tuple[int, ...] | RedisFailure: ...
|
||||
|
||||
async def subscribe(self, *channels: str) -> RedisSubscription | RedisFailure: ...
|
||||
|
||||
async def ping(self) -> bool | RedisFailure: ...
|
||||
|
||||
async def info(self) -> Mapping[str, InfoValue] | RedisFailure: ...
|
||||
|
||||
async def client_list(self) -> tuple[Mapping[str, str], ...] | RedisFailure: ...
|
||||
|
||||
async def flushall(self) -> bool | RedisFailure: ...
|
||||
109
litellm/redis/transport/events.py
Normal file
109
litellm/redis/transport/events.py
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
from collections.abc import Awaitable, Callable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Final, TypeVar
|
||||
|
||||
from litellm._internal_context import current_service_target, service_target
|
||||
from litellm._logging import verbose_logger
|
||||
from litellm._service_logger import ServiceLogging
|
||||
from litellm.caching.redis_cache import (
|
||||
_get_call_stack_info, # pyright: ignore[reportPrivateUsage] # same caller attribution as RedisCache events
|
||||
log_redis_failure,
|
||||
log_redis_failure_value,
|
||||
)
|
||||
from litellm.redis.transport.contract import RedisFailure
|
||||
from litellm.types.services import ServiceTypes
|
||||
|
||||
_T: Final = TypeVar("_T")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RedisServiceEvents:
|
||||
service_logging: ServiceLogging
|
||||
|
||||
async def observe(self, call_type: str, call: Callable[[], Awaitable[_T | RedisFailure]]) -> _T | RedisFailure:
|
||||
if not self.service_logging.has_sinks():
|
||||
result: Final = await call()
|
||||
_log_failure(call_type, result)
|
||||
return result
|
||||
caller: Final = _get_call_stack_info()
|
||||
start_time: Final = time.time()
|
||||
observed: Final = await call()
|
||||
end_time: Final = time.time()
|
||||
_log_failure(call_type, observed)
|
||||
await self._emit(call_type, observed, caller=caller, start_time=start_time, end_time=end_time)
|
||||
return observed
|
||||
|
||||
async def _emit(self, call_type: str, result: object, *, caller: str, start_time: float, end_time: float) -> None:
|
||||
if not isinstance(result, RedisFailure):
|
||||
await ServiceLogging.emit_guarded(
|
||||
lambda: self.service_logging.async_service_success_hook(
|
||||
service=ServiceTypes.REDIS,
|
||||
call_type=call_type,
|
||||
duration=end_time - start_time,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
caller=caller,
|
||||
)
|
||||
)
|
||||
return
|
||||
if result.kind == "breaker_open":
|
||||
return
|
||||
await ServiceLogging.emit_guarded(
|
||||
lambda: self.service_logging.async_service_failure_hook(
|
||||
service=ServiceTypes.REDIS,
|
||||
duration=end_time - start_time,
|
||||
error=result.message,
|
||||
call_type=call_type,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
caller=caller,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _log_failure(call_type: str, result: object) -> None:
|
||||
if not isinstance(result, RedisFailure):
|
||||
return
|
||||
target: Final = current_service_target()
|
||||
log_redis_failure_value(
|
||||
verbose_logger, logging.WARNING, f"Redis {call_type} failed" + (f" for {target}" if target else ""), result
|
||||
)
|
||||
|
||||
|
||||
def report_pipeline(
|
||||
service_logging: ServiceLogging,
|
||||
call_type: str,
|
||||
*,
|
||||
op_count: int,
|
||||
target: str | None,
|
||||
metadata: Mapping[str, int | str],
|
||||
start_time: float,
|
||||
error: Exception | None,
|
||||
) -> None:
|
||||
end_time: Final = time.time()
|
||||
if error is not None:
|
||||
log_redis_failure(verbose_logger, logging.WARNING, f"{call_type}: pipeline of {op_count} ops failed", error)
|
||||
with service_target(target):
|
||||
asyncio.create_task(
|
||||
service_logging.async_service_success_hook(
|
||||
service=ServiceTypes.REDIS,
|
||||
duration=end_time - start_time,
|
||||
call_type=call_type,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
event_metadata=dict(metadata),
|
||||
)
|
||||
if error is None
|
||||
else service_logging.async_service_failure_hook(
|
||||
service=ServiceTypes.REDIS,
|
||||
duration=end_time - start_time,
|
||||
error=error,
|
||||
call_type=call_type,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
event_metadata=dict(metadata),
|
||||
)
|
||||
)
|
||||
584
litellm/redis/transport/redis_py.py
Normal file
584
litellm/redis/transport/redis_py.py
Normal file
|
|
@ -0,0 +1,584 @@
|
|||
import asyncio
|
||||
import hashlib
|
||||
from collections.abc import Awaitable, Callable, Iterator, Mapping, Sequence
|
||||
from dataclasses import dataclass
|
||||
from datetime import timedelta
|
||||
from types import MappingProxyType
|
||||
from typing import Final, Generic, TypeVar
|
||||
|
||||
from pydantic import TypeAdapter, ValidationError
|
||||
from redis.exceptions import NoScriptError
|
||||
|
||||
from litellm.caching.redis_cache import RedisCircuitBreaker
|
||||
from litellm.redis.transport.batch import BatchResult, PipelineBackend, PipelineBatch
|
||||
from litellm.redis.transport.breaker import failure_from, run_under_breaker
|
||||
from litellm.redis.transport.client import AsyncPubSub, AsyncPubSubClient, AsyncRedisCommands, RedisPipeline
|
||||
from litellm.redis.transport.contract import (
|
||||
Admit,
|
||||
Arg,
|
||||
EvalReply,
|
||||
InfoValue,
|
||||
Popped,
|
||||
PubSubMessage,
|
||||
RedisFailure,
|
||||
ttl_ms,
|
||||
)
|
||||
from litellm.redis.transport.events import RedisServiceEvents
|
||||
|
||||
_T: Final = TypeVar("_T")
|
||||
|
||||
_ARRAY: Final = TypeAdapter(tuple[object, ...])
|
||||
_INFO: Final[TypeAdapter[Mapping[str, InfoValue]]] = TypeAdapter(Mapping[str, InfoValue])
|
||||
_CLIENTS: Final = TypeAdapter(tuple[Mapping[str, str], ...])
|
||||
_MESSAGE: Final = TypeAdapter(Mapping[str, object])
|
||||
|
||||
|
||||
def is_missing_script(failure: Exception) -> bool:
|
||||
return isinstance(failure, NoScriptError)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RedisPyTransport:
|
||||
client: Callable[[], AsyncRedisCommands]
|
||||
namespace: str | None
|
||||
breaker: RedisCircuitBreaker
|
||||
admit: Admit
|
||||
events: RedisServiceEvents
|
||||
multi_get: Callable[[Sequence[str]], Awaitable[object]]
|
||||
server_version: Callable[[], int]
|
||||
pubsub_client: Callable[[], AsyncPubSubClient]
|
||||
request_engine: Callable[[], PipelineBatch | None]
|
||||
post_call_engine: Callable[[], PipelineBatch | None]
|
||||
|
||||
async def get(self, key: str) -> bytes | None | RedisFailure:
|
||||
return _bytes_reply(await self._run("get", lambda: self.client().get(self._key(key))))
|
||||
|
||||
async def mget(self, keys: Sequence[str]) -> tuple[bytes | None, ...] | RedisFailure:
|
||||
namespaced: Final = tuple(self._key(k) for k in keys)
|
||||
return _mget_reply(await self._run("mget", lambda: self.multi_get(namespaced)), len(namespaced))
|
||||
|
||||
async def set(self, key: str, value: Arg, *, ttl: timedelta | None, nx: bool = False) -> bool | RedisFailure:
|
||||
px: Final = None if ttl is None else ttl_ms(ttl)
|
||||
return _bool_reply(await self._run("set", lambda: self.client().set(self._key(key), value, px=px, nx=nx)))
|
||||
|
||||
async def delete(self, *keys: str) -> int | RedisFailure:
|
||||
namespaced: Final = tuple(self._key(k) for k in keys)
|
||||
return _int_reply(await self._run("delete", lambda: self.client().delete(*namespaced)))
|
||||
|
||||
async def incrby(self, key: str, amount: int) -> int | RedisFailure:
|
||||
return _int_reply(await self._run("incrby", lambda: self.client().incrby(self._key(key), amount)))
|
||||
|
||||
async def incrbyfloat(self, key: str, amount: float) -> float | RedisFailure:
|
||||
return _float_reply(await self._run("incrbyfloat", lambda: self.client().incrbyfloat(self._key(key), amount)))
|
||||
|
||||
async def expire(self, key: str, ttl: timedelta) -> bool | RedisFailure:
|
||||
return _bool_reply(await self._run("expire", lambda: self.client().pexpire(self._key(key), ttl_ms(ttl))))
|
||||
|
||||
async def ttl(self, key: str) -> int | RedisFailure:
|
||||
return _int_reply(await self._run("ttl", lambda: self.client().ttl(self._key(key))))
|
||||
|
||||
async def rpush(self, key: str, *values: Arg) -> int | RedisFailure:
|
||||
return _int_reply(await self._run("rpush", lambda: self.client().rpush(self._key(key), *values)))
|
||||
|
||||
async def lpop(self, key: str, count: int | None = None) -> Popped | RedisFailure:
|
||||
name: Final = self._key(key)
|
||||
if count is None or self.server_version() >= 7:
|
||||
return _popped_reply(await self._run("lpop", lambda: self.client().lpop(name, count)))
|
||||
return _popped_each(await self._run("lpop", lambda: _lpop_each_alone(self.client(), name, count)))
|
||||
|
||||
async def ltrim(self, key: str, start: int, stop: int) -> bool | RedisFailure:
|
||||
return _bool_reply(await self._run("ltrim", lambda: self.client().ltrim(self._key(key), start, stop)))
|
||||
|
||||
async def sadd(self, key: str, *members: Arg) -> int | RedisFailure:
|
||||
return _int_reply(await self._run("sadd", lambda: self.client().sadd(self._key(key), *members)))
|
||||
|
||||
async def scan(self, prefix: str, *, limit: int) -> tuple[bytes, ...] | RedisFailure:
|
||||
match: Final = f"{self._key(prefix)}*"
|
||||
return _keys_reply(await self._run("scan", lambda: _scan(self.client(), match, limit)))
|
||||
|
||||
async def eval(self, script: str, keys: Sequence[str], args: Sequence[Arg]) -> EvalReply | RedisFailure:
|
||||
sha: Final = _sha(script)
|
||||
namespaced: Final = tuple(self._key(k) for k in keys)
|
||||
return _eval_reply(
|
||||
await self._run("eval", lambda: _evalsha_or_eval(self.client(), script, sha, namespaced, args))
|
||||
)
|
||||
|
||||
def batch(self, *, transaction: bool = False) -> "RedisPyCommandBatch":
|
||||
return RedisPyCommandBatch(self, PipelineBatch(self._backend(runs_alone=False), transaction=transaction))
|
||||
|
||||
def request_batch(self) -> "RedisPyCommandBatch | None":
|
||||
engine: Final = self.request_engine()
|
||||
return None if engine is None else RedisPyCommandBatch(self, engine)
|
||||
|
||||
def post_call_batch(self) -> "RedisPyCommandBatch | None":
|
||||
engine: Final = self.post_call_engine()
|
||||
return None if engine is None else RedisPyCommandBatch(self, engine)
|
||||
|
||||
async def publish(self, channel: str, message: str | bytes) -> int | RedisFailure:
|
||||
return _int_reply(
|
||||
await self._run_unguarded("publish", lambda: self.pubsub_client().publish(self._key(channel), message))
|
||||
)
|
||||
|
||||
async def publish_all(self, channel: str, messages: Sequence[str | bytes]) -> tuple[int, ...] | RedisFailure:
|
||||
name: Final = self._key(channel)
|
||||
return _ints_reply(
|
||||
await self._run_unguarded("publish", lambda: _publish_each(self.pubsub_client(), name, messages)),
|
||||
len(messages),
|
||||
)
|
||||
|
||||
async def subscribe(self, *channels: str) -> "RedisPySubscription | RedisFailure":
|
||||
names: Final = tuple(self._key(channel) for channel in channels)
|
||||
return await self._run_unguarded("subscribe", lambda: _subscribe(self.pubsub_client(), names))
|
||||
|
||||
async def ping(self) -> bool | RedisFailure:
|
||||
return _bool_reply(await self._run_unguarded("ping", lambda: self.client().ping()))
|
||||
|
||||
async def info(self) -> Mapping[str, InfoValue] | RedisFailure:
|
||||
return _info_reply(await self._run_unguarded("info", lambda: self.client().info()))
|
||||
|
||||
async def client_list(self) -> tuple[Mapping[str, str], ...] | RedisFailure:
|
||||
return _clients_reply(await self._run_unguarded("client_list", lambda: self.client().client_list()))
|
||||
|
||||
async def flushall(self) -> bool | RedisFailure:
|
||||
return _bool_reply(await self._run_unguarded("flushall", lambda: self.client().flushall()))
|
||||
|
||||
def _key(self, key: str) -> str:
|
||||
if not self.namespace or key.startswith(f"{self.namespace}:"):
|
||||
return key
|
||||
return f"{self.namespace}:{key}"
|
||||
|
||||
def _backend(self, *, runs_alone: bool) -> PipelineBackend:
|
||||
return PipelineBackend(
|
||||
client=self.client,
|
||||
breaker=self.breaker,
|
||||
admit=self.admit,
|
||||
service_logging=self.events.service_logging,
|
||||
runs_alone=runs_alone,
|
||||
)
|
||||
|
||||
async def _run(self, name: str, call: Callable[[], Awaitable[_T]]) -> _T | RedisFailure:
|
||||
return await self.events.observe(name, lambda: run_under_breaker(self.breaker, name, self.admit, call))
|
||||
|
||||
async def _run_unguarded(self, name: str, call: Callable[[], Awaitable[_T]]) -> _T | RedisFailure:
|
||||
return await self.events.observe(name, lambda: _without_breaker(call))
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _Command(Generic[_T]):
|
||||
queue: Callable[[RedisPipeline], int]
|
||||
decode: Callable[[Sequence[object]], _T | RedisFailure]
|
||||
alone: Callable[[], Awaitable[_T | RedisFailure]]
|
||||
|
||||
def enqueue(self, pipe: RedisPipeline) -> int:
|
||||
return self.queue(pipe)
|
||||
|
||||
def resolve(self, replies: Sequence[object]) -> _T | RedisFailure:
|
||||
return self.decode(replies)
|
||||
|
||||
def recover(self, error: Exception) -> _T | RedisFailure:
|
||||
failure: Final = failure_from(error)
|
||||
if failure is None:
|
||||
raise error
|
||||
return failure
|
||||
|
||||
def run_alone(self) -> Awaitable[_T | RedisFailure]:
|
||||
return self.alone()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RedisPyCommandBatch:
|
||||
transport: RedisPyTransport
|
||||
engine: PipelineBatch
|
||||
|
||||
def get(self, key: str) -> BatchResult[bytes | None | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
reply: Final = self._declare(
|
||||
lambda pipe: _one(pipe.get(name)), lambda replies: _bytes_reply(replies[0]), lambda: self.transport.get(key)
|
||||
)
|
||||
reply.on_settled(lambda future: self._note_read((key,), future, lambda value: (value,)))
|
||||
return reply
|
||||
|
||||
def mget(self, keys: Sequence[str]) -> BatchResult[tuple[bytes | None, ...] | RedisFailure]:
|
||||
logical: Final = tuple(keys)
|
||||
names: Final = tuple(self.transport._key(k) for k in logical) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
reply: Final = self._declare(
|
||||
lambda pipe: _one(pipe.mget(names)),
|
||||
lambda replies: _mget_reply(replies[0], len(names)),
|
||||
lambda: self.transport.mget(logical),
|
||||
)
|
||||
reply.on_settled(lambda future: self._note_read(logical, future, lambda values: values))
|
||||
return reply
|
||||
|
||||
def set(self, key: str, value: Arg, *, ttl: timedelta | None, nx: bool = False) -> BatchResult[bool | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
px: Final = None if ttl is None else ttl_ms(ttl)
|
||||
self.engine.note_present(key)
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.set(name, value, px=px, nx=nx)),
|
||||
lambda replies: _bool_reply(replies[0]),
|
||||
lambda: self.transport.set(key, value, ttl=ttl, nx=nx),
|
||||
)
|
||||
|
||||
def delete(self, *keys: str) -> BatchResult[int | RedisFailure]:
|
||||
names: Final = tuple(self.transport._key(k) for k in keys) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
self.engine.note_missing(keys)
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.delete(*names)),
|
||||
lambda replies: _int_reply(replies[0]),
|
||||
lambda: self.transport.delete(*keys),
|
||||
)
|
||||
|
||||
def incrby(self, key: str, amount: int) -> BatchResult[int | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
self.engine.note_present(key)
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.incrby(name, amount)),
|
||||
lambda replies: _int_reply(replies[0]),
|
||||
lambda: self.transport.incrby(key, amount),
|
||||
)
|
||||
|
||||
def incrbyfloat(self, key: str, amount: float) -> BatchResult[float | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
self.engine.note_present(key)
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.incrbyfloat(name, amount)),
|
||||
lambda replies: _float_reply(replies[0]),
|
||||
lambda: self.transport.incrbyfloat(key, amount),
|
||||
)
|
||||
|
||||
def expire(self, key: str, ttl: timedelta) -> BatchResult[bool | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.pexpire(name, ttl_ms(ttl))),
|
||||
lambda replies: _bool_reply(replies[0]),
|
||||
lambda: self.transport.expire(key, ttl),
|
||||
)
|
||||
|
||||
def ttl(self, key: str) -> BatchResult[int | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.ttl(name)), lambda replies: _int_reply(replies[0]), lambda: self.transport.ttl(key)
|
||||
)
|
||||
|
||||
def rpush(self, key: str, *values: Arg) -> BatchResult[int | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
self.engine.note_present(key)
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.rpush(name, *values)),
|
||||
lambda replies: _int_reply(replies[0]),
|
||||
lambda: self.transport.rpush(key, *values),
|
||||
)
|
||||
|
||||
def lpop(self, key: str, count: int | None = None) -> BatchResult[Popped | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
if count is None or self.transport.server_version() >= 7:
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.lpop(name, count)),
|
||||
lambda replies: _popped_reply(replies[0]),
|
||||
lambda: self.transport.lpop(key, count),
|
||||
)
|
||||
return self._declare(
|
||||
lambda pipe: _queue_lpop_each(pipe, name, count),
|
||||
_popped_each,
|
||||
lambda: self.transport.lpop(key, count),
|
||||
)
|
||||
|
||||
def ltrim(self, key: str, start: int, stop: int) -> BatchResult[bool | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.ltrim(name, start, stop)),
|
||||
lambda replies: _bool_reply(replies[0]),
|
||||
lambda: self.transport.ltrim(key, start, stop),
|
||||
)
|
||||
|
||||
def sadd(self, key: str, *members: Arg) -> BatchResult[int | RedisFailure]:
|
||||
name: Final = self.transport._key(key) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
self.engine.note_present(key)
|
||||
return self._declare(
|
||||
lambda pipe: _one(pipe.sadd(name, *members)),
|
||||
lambda replies: _int_reply(replies[0]),
|
||||
lambda: self.transport.sadd(key, *members),
|
||||
)
|
||||
|
||||
def eval(self, script: str, keys: Sequence[str], args: Sequence[Arg]) -> BatchResult[EvalReply | RedisFailure]:
|
||||
names: Final = tuple(self.transport._key(k) for k in keys) # pyright: ignore[reportPrivateUsage] # the batch is the transport's own
|
||||
sha: Final = _sha(script)
|
||||
transaction: Final = self.engine.transaction
|
||||
return self._declare(
|
||||
lambda pipe: _one(
|
||||
pipe.eval(script, len(names), *names, *args)
|
||||
if transaction
|
||||
else pipe.evalsha(sha, len(names), *names, *args)
|
||||
),
|
||||
lambda replies: _eval_reply(replies[0]),
|
||||
lambda: self.transport.eval(script, keys, args),
|
||||
)
|
||||
|
||||
async def flush(self) -> None:
|
||||
await self.engine.flush()
|
||||
|
||||
def _declare(
|
||||
self,
|
||||
queue: Callable[[RedisPipeline], int],
|
||||
decode: Callable[[Sequence[object]], _T | RedisFailure],
|
||||
alone: Callable[[], Awaitable[_T | RedisFailure]],
|
||||
) -> BatchResult[_T | RedisFailure]:
|
||||
return self.engine.declare(_Command(queue, decode, alone))
|
||||
|
||||
def _note_read(
|
||||
self,
|
||||
keys: Sequence[str],
|
||||
future: "asyncio.Future[_T | RedisFailure]",
|
||||
values: Callable[[_T], Sequence[bytes | None]],
|
||||
) -> None:
|
||||
if future.cancelled() or future.exception() is not None:
|
||||
return
|
||||
result: Final = future.result()
|
||||
if isinstance(result, RedisFailure):
|
||||
return
|
||||
self.engine.note_missing(key for key, value in zip(keys, values(result)) if value is None)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RedisPySubscription:
|
||||
pubsub: AsyncPubSub
|
||||
|
||||
async def get_message(self, *, timeout: float) -> PubSubMessage | None | RedisFailure:
|
||||
return _message_reply(
|
||||
await _without_breaker(lambda: self.pubsub.get_message(ignore_subscribe_messages=True, timeout=timeout))
|
||||
)
|
||||
|
||||
async def close(self) -> None | RedisFailure:
|
||||
closed: Final = await _without_breaker(self.pubsub.reset)
|
||||
return closed if isinstance(closed, RedisFailure) else None
|
||||
|
||||
|
||||
def _one(_queued: object) -> int:
|
||||
return 1
|
||||
|
||||
|
||||
def _sha(script: str) -> str:
|
||||
return hashlib.sha1(script.encode(), usedforsecurity=False).hexdigest()
|
||||
|
||||
|
||||
def _queue_lpop_each(pipe: RedisPipeline, name: str, count: int) -> int:
|
||||
for _ in range(count):
|
||||
pipe.lpop(name)
|
||||
return count
|
||||
|
||||
|
||||
async def _lpop_each_alone(client: AsyncRedisCommands, name: str, count: int) -> Sequence[object]:
|
||||
async with client.pipeline(transaction=False) as pipe:
|
||||
_queue_lpop_each(pipe, name, count)
|
||||
return await pipe.execute()
|
||||
|
||||
|
||||
async def _publish_each(client: AsyncPubSubClient, channel: str, messages: Sequence[str | bytes]) -> Sequence[object]:
|
||||
async with client.pipeline(transaction=False) as pipe:
|
||||
for message in messages:
|
||||
pipe.publish(channel, message)
|
||||
return await pipe.execute()
|
||||
|
||||
|
||||
async def _subscribe(client: AsyncPubSubClient, channels: Sequence[str]) -> RedisPySubscription:
|
||||
pubsub: Final = client.pubsub()
|
||||
await pubsub.subscribe(*channels)
|
||||
return RedisPySubscription(pubsub)
|
||||
|
||||
|
||||
async def _scan(client: AsyncRedisCommands, match: str, limit: int) -> tuple[object, ...]:
|
||||
keys: Final[list[object]] = [] # mutable-ok: filled while the cursor walks
|
||||
async for key in client.scan_iter(match=match, count=limit):
|
||||
keys.append(key)
|
||||
if len(keys) >= limit:
|
||||
break
|
||||
return tuple(keys)
|
||||
|
||||
|
||||
async def _without_breaker(call: Callable[[], Awaitable[_T]]) -> _T | RedisFailure:
|
||||
try:
|
||||
return await call()
|
||||
except Exception as e:
|
||||
failure: Final = failure_from(e)
|
||||
if failure is None:
|
||||
raise
|
||||
return failure
|
||||
|
||||
|
||||
async def _evalsha_or_eval(
|
||||
client: AsyncRedisCommands, script: str, sha: str, keys: Sequence[str], args: Sequence[Arg]
|
||||
) -> object:
|
||||
try:
|
||||
return await client.evalsha(sha, len(keys), *keys, *args)
|
||||
except NoScriptError:
|
||||
return await client.eval(script, len(keys), *keys, *args)
|
||||
|
||||
|
||||
def _decode_failure(reply: object) -> RedisFailure:
|
||||
return RedisFailure(kind="decode", message=f"unexpected Redis reply of type {type(reply).__name__}")
|
||||
|
||||
|
||||
def _optional_bytes(reply: object) -> bytes | None | RedisFailure:
|
||||
if reply is None or isinstance(reply, bytes):
|
||||
return reply
|
||||
if isinstance(reply, str):
|
||||
return reply.encode()
|
||||
return _decode_failure(reply)
|
||||
|
||||
|
||||
def _bytes_reply(reply: object) -> bytes | None | RedisFailure:
|
||||
return reply if isinstance(reply, RedisFailure) else _optional_bytes(reply)
|
||||
|
||||
|
||||
def _array(reply: object) -> tuple[object, ...] | None:
|
||||
if not isinstance(reply, (list, tuple)):
|
||||
return None
|
||||
try:
|
||||
return _ARRAY.validate_python(reply)
|
||||
except ValidationError:
|
||||
return None
|
||||
|
||||
|
||||
def _all_bytes(values: Sequence[object]) -> tuple[bytes, ...] | RedisFailure:
|
||||
decoded: Final = tuple(_optional_bytes(value) for value in values)
|
||||
failure: Final = next((value for value in decoded if isinstance(value, RedisFailure)), None)
|
||||
if failure is not None:
|
||||
return failure
|
||||
present: Final = tuple(value for value in decoded if isinstance(value, bytes))
|
||||
return present if len(present) == len(decoded) else _decode_failure(values)
|
||||
|
||||
|
||||
def _mget_reply(reply: object, expected: int) -> tuple[bytes | None, ...] | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
array: Final = _array(reply)
|
||||
if array is None or len(array) != expected:
|
||||
return _decode_failure(reply)
|
||||
values: Final = tuple(_optional_bytes(value) for value in array)
|
||||
failure: Final = next((value for value in values if isinstance(value, RedisFailure)), None)
|
||||
return failure if failure is not None else tuple(value for value in values if not isinstance(value, RedisFailure))
|
||||
|
||||
|
||||
def _keys_reply(reply: object) -> tuple[bytes, ...] | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
array: Final = _array(reply)
|
||||
return _decode_failure(reply) if array is None else _all_bytes(array)
|
||||
|
||||
|
||||
def _popped_reply(reply: object) -> Popped | RedisFailure:
|
||||
if isinstance(reply, RedisFailure) or reply is None:
|
||||
return reply
|
||||
array: Final = _array(reply)
|
||||
if array is None:
|
||||
return _bytes_reply(reply)
|
||||
return _all_bytes(array)
|
||||
|
||||
|
||||
def _popped_each(replies: object) -> Popped | RedisFailure:
|
||||
if isinstance(replies, RedisFailure):
|
||||
return replies
|
||||
array: Final = _array(replies)
|
||||
if array is None:
|
||||
return _decode_failure(replies)
|
||||
popped: Final = _all_bytes(tuple(value for value in array if value is not None))
|
||||
return None if popped == () else popped
|
||||
|
||||
|
||||
def _bool_reply(reply: object) -> bool | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
return reply is not None and reply is not False and reply != 0
|
||||
|
||||
|
||||
def _int_reply(reply: object) -> int | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
return reply if isinstance(reply, int) and not isinstance(reply, bool) else _decode_failure(reply)
|
||||
|
||||
|
||||
def _ints_reply(reply: object, expected: int) -> tuple[int, ...] | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
array: Final = _array(reply)
|
||||
if array is None or len(array) != expected:
|
||||
return _decode_failure(reply)
|
||||
counts: Final = tuple(_int_reply(value) for value in array)
|
||||
failure: Final = next((value for value in counts if isinstance(value, RedisFailure)), None)
|
||||
return failure if failure is not None else tuple(value for value in counts if not isinstance(value, RedisFailure))
|
||||
|
||||
|
||||
def _float_reply(reply: object) -> float | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
if isinstance(reply, bool) or not isinstance(reply, (int, float, str, bytes)):
|
||||
return _decode_failure(reply)
|
||||
try:
|
||||
return float(reply)
|
||||
except ValueError:
|
||||
return _decode_failure(reply)
|
||||
|
||||
|
||||
def _info_reply(reply: object) -> Mapping[str, InfoValue] | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
try:
|
||||
return MappingProxyType(_INFO.validate_python(reply))
|
||||
except ValidationError:
|
||||
return _decode_failure(reply)
|
||||
|
||||
|
||||
def _clients_reply(reply: object) -> tuple[Mapping[str, str], ...] | RedisFailure:
|
||||
if isinstance(reply, RedisFailure):
|
||||
return reply
|
||||
try:
|
||||
return tuple(MappingProxyType(client) for client in _CLIENTS.validate_python(reply))
|
||||
except ValidationError:
|
||||
return _decode_failure(reply)
|
||||
|
||||
|
||||
def _message_reply(reply: object) -> PubSubMessage | None | RedisFailure:
|
||||
if isinstance(reply, RedisFailure) or reply is None:
|
||||
return reply
|
||||
try:
|
||||
message: Final = _MESSAGE.validate_python(reply)
|
||||
except ValidationError:
|
||||
return _decode_failure(reply)
|
||||
channel: Final = _optional_bytes(message.get("channel"))
|
||||
data: Final = _optional_bytes(message.get("data"))
|
||||
if not isinstance(channel, bytes) or not isinstance(data, bytes):
|
||||
return _decode_failure(reply)
|
||||
return PubSubMessage(channel=channel, data=data)
|
||||
|
||||
|
||||
def _eval_reply(reply: object) -> EvalReply | RedisFailure:
|
||||
frames: Final[list[tuple[Iterator[object], list[EvalReply]]]] = [ # mutable-ok: linear-time traversal stack
|
||||
(iter((reply,)), [])
|
||||
]
|
||||
while frames:
|
||||
pending, values = frames[-1]
|
||||
for item in pending:
|
||||
if (array := _array(item)) is not None:
|
||||
frames.append((iter(array), []))
|
||||
break
|
||||
if isinstance(scalar := _eval_scalar(item), RedisFailure):
|
||||
return scalar
|
||||
values.append(scalar)
|
||||
else:
|
||||
frames.pop()
|
||||
if not frames:
|
||||
return values[0]
|
||||
frames[-1][1].append(tuple(values))
|
||||
return _decode_failure(reply)
|
||||
|
||||
|
||||
def _eval_scalar(reply: object) -> int | bytes | None | RedisFailure:
|
||||
if isinstance(reply, RedisFailure) or reply is None or isinstance(reply, bytes):
|
||||
return reply
|
||||
if isinstance(reply, bool):
|
||||
return _decode_failure(reply)
|
||||
if isinstance(reply, int):
|
||||
return reply
|
||||
if isinstance(reply, str):
|
||||
return reply.encode()
|
||||
return _decode_failure(reply)
|
||||
|
|
@ -233,7 +233,7 @@ dev = [
|
|||
"opentelemetry-instrumentation-fastapi==0.66b1",
|
||||
"langfuse>=4.7,<5.0",
|
||||
"fastapi-offline==1.7.6",
|
||||
"fakeredis==2.34.1",
|
||||
"fakeredis[lua]==2.34.1",
|
||||
"pytest-rerunfailures==15.1",
|
||||
"pytest-cov==5.0.0",
|
||||
"parameterized==0.9.0",
|
||||
|
|
|
|||
54
tests/_support/fake_redis.py
Normal file
54
tests/_support/fake_redis.py
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
from collections.abc import Awaitable, Callable
|
||||
from typing import TypeVar
|
||||
|
||||
import fakeredis
|
||||
|
||||
import litellm
|
||||
from litellm._service_logger import ServiceLogging
|
||||
from litellm.caching.redis_cache import RedisCache, RedisCircuitBreaker
|
||||
from litellm.redis.transport.events import RedisServiceEvents
|
||||
from litellm.redis.transport.redis_py import RedisPyTransport
|
||||
|
||||
_T = TypeVar("_T")
|
||||
|
||||
|
||||
def _admit_directly(call: Callable[[], Awaitable[_T]]) -> Awaitable[_T]:
|
||||
return call()
|
||||
|
||||
|
||||
def fake_redis_cache(
|
||||
server: fakeredis.FakeServer, namespace: str | None = None, service_logger: ServiceLogging | None = None
|
||||
) -> RedisCache:
|
||||
cache = RedisCache(
|
||||
host="127.0.0.1",
|
||||
port=1,
|
||||
namespace=namespace,
|
||||
socket_timeout=0.1,
|
||||
service_logger_obj=service_logger or ServiceLogging(),
|
||||
)
|
||||
litellm.in_memory_llm_clients_cache.set_cache(
|
||||
key=cache._get_async_client_cache_key(), # pyright: ignore[reportPrivateUsage] # seeds the per-loop client slot RedisCache reads
|
||||
value=fakeredis.FakeAsyncRedis(server=server),
|
||||
)
|
||||
return cache
|
||||
|
||||
|
||||
def fake_redis_transport(
|
||||
client: fakeredis.FakeAsyncRedis,
|
||||
*,
|
||||
failure_threshold: int,
|
||||
events: ServiceLogging | None = None,
|
||||
namespace: str | None = None,
|
||||
) -> RedisPyTransport:
|
||||
return RedisPyTransport(
|
||||
client=lambda: client,
|
||||
namespace=namespace,
|
||||
breaker=RedisCircuitBreaker(failure_threshold=failure_threshold, recovery_timeout=3600),
|
||||
admit=_admit_directly,
|
||||
events=RedisServiceEvents(events or ServiceLogging()),
|
||||
multi_get=lambda keys: client.mget(keys),
|
||||
server_version=lambda: 7,
|
||||
pubsub_client=lambda: client,
|
||||
request_engine=lambda: None,
|
||||
post_call_engine=lambda: None,
|
||||
)
|
||||
0
tests/unit/redis/__init__.py
Normal file
0
tests/unit/redis/__init__.py
Normal file
12
tests/unit/redis/conftest.py
Normal file
12
tests/unit/redis/conftest.py
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
import fakeredis
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def server() -> fakeredis.FakeServer:
|
||||
return fakeredis.FakeServer()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def raw(server: fakeredis.FakeServer) -> fakeredis.FakeAsyncRedis:
|
||||
return fakeredis.FakeAsyncRedis(server=server)
|
||||
0
tests/unit/redis/transport/__init__.py
Normal file
0
tests/unit/redis/transport/__init__.py
Normal file
17
tests/unit/redis/transport/test_contract.py
Normal file
17
tests/unit/redis/transport/test_contract.py
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
from datetime import timedelta
|
||||
|
||||
import pytest
|
||||
|
||||
from litellm.redis.transport.contract import ttl_ms
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("ttl", "expected"),
|
||||
[
|
||||
pytest.param(timedelta(seconds=1.5), 1500, id="whole-ms"),
|
||||
pytest.param(timedelta(microseconds=1500), 2, id="rounds-up"),
|
||||
pytest.param(timedelta(microseconds=1), 1, id="sub-ms-is-not-zero"),
|
||||
],
|
||||
)
|
||||
def test_ttl_ms_never_shortens_the_ttl_or_sends_zero(ttl: timedelta, expected: int) -> None:
|
||||
assert ttl_ms(ttl) == expected
|
||||
718
tests/unit/redis/transport/test_redis_py.py
Normal file
718
tests/unit/redis/transport/test_redis_py.py
Normal file
|
|
@ -0,0 +1,718 @@
|
|||
import asyncio
|
||||
import logging
|
||||
from collections.abc import Awaitable, Callable, Sequence
|
||||
from dataclasses import dataclass, replace
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Final, Literal
|
||||
from unittest.mock import patch
|
||||
|
||||
import fakeredis
|
||||
import pytest
|
||||
from redis.exceptions import ResponseError
|
||||
|
||||
import litellm
|
||||
from litellm._internal_context import current_service_target, service_target
|
||||
from litellm._service_logger import ServiceLogging
|
||||
from litellm.caching.redis_batch import active_request_redis_batch, request_redis_batch_scope
|
||||
from litellm.caching.redis_cluster_cache import ClusterConnectionGate, RedisClusterCache
|
||||
from litellm.redis.transport.batch import PipelineBatch
|
||||
from litellm.redis.transport.contract import PubSubMessage, RedisFailure
|
||||
from litellm.redis.transport.redis_py import RedisPyCommandBatch, RedisPyTransport
|
||||
from litellm.types.services import ServiceTypes
|
||||
from tests._support.fake_redis import fake_redis_cache, fake_redis_transport
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_every_command_prefixes_the_namespace_once(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
|
||||
assert await transport.set("k", "v", ttl=None) is True
|
||||
assert await transport.set("ns:gone", "v", ttl=None) is True
|
||||
|
||||
assert sorted(await raw.keys("*")) == [b"ns:gone", b"ns:k"]
|
||||
assert await transport.get("k") == await transport.get("ns:k") == b"v"
|
||||
assert await transport.delete("gone") == 1
|
||||
assert await raw.keys("*") == [b"ns:k"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_nx_refuses_a_live_key_and_applies_the_ttl(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
|
||||
assert await transport.set("k", "first", ttl=timedelta(seconds=30), nx=True) is True
|
||||
assert await transport.set("k", "second", ttl=timedelta(seconds=30), nx=True) is False
|
||||
|
||||
assert await raw.get("k") == b"first"
|
||||
assert 0 < await raw.pttl("k") <= 30_000
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_without_ttl_persists(server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
|
||||
assert await transport.set("k", "v", ttl=None) is True
|
||||
|
||||
assert await raw.pttl("k") == -1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_reports_how_many_keys_it_removed(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await raw.set("k", "v")
|
||||
|
||||
assert await transport.delete("k") == 1
|
||||
assert await transport.delete("k") == 0
|
||||
assert await raw.exists("k") == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_rejected_command_is_a_server_failure_and_keeps_the_connection(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await raw.rpush("list", "x")
|
||||
|
||||
rejected = await transport.get("list")
|
||||
|
||||
assert isinstance(rejected, RedisFailure)
|
||||
assert rejected.kind == "server"
|
||||
assert await transport.set("k", "v", ttl=None) is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_unreachable_server_is_a_transport_failure(server: fakeredis.FakeServer) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
server.connected = False
|
||||
|
||||
failure = await transport.get("k")
|
||||
|
||||
assert isinstance(failure, RedisFailure)
|
||||
assert failure.kind == "transport"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_transport_failure_opens_the_breaker_and_it_answers_without_redis(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_transport(raw, failure_threshold=1)
|
||||
server.connected = False
|
||||
|
||||
tripped = await transport.get("k")
|
||||
server.connected = True
|
||||
refused = await transport.get("k")
|
||||
|
||||
assert isinstance(tripped, RedisFailure)
|
||||
assert tripped.kind == "transport"
|
||||
assert isinstance(refused, RedisFailure)
|
||||
assert refused.kind == "breaker_open"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mget_returns_one_slot_per_key_in_order(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
await raw.set("ns:a", "1")
|
||||
await raw.set("ns:c", "3")
|
||||
|
||||
assert await transport.mget(("a", "b", "ns:c")) == (b"1", None, b"3")
|
||||
assert await transport.mget(()) == ()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_delete_takes_several_keys(server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
await raw.set("ns:a", "1")
|
||||
await raw.set("ns:b", "2")
|
||||
await raw.set("c", "3")
|
||||
|
||||
assert await transport.delete("a", "b", "c") == 2
|
||||
assert await raw.keys("*") == [b"c"]
|
||||
|
||||
|
||||
_ECHO: Final = """
|
||||
redis.call("set", KEYS[1], ARGV[1])
|
||||
if ARGV[2] == "false" then
|
||||
return false
|
||||
end
|
||||
return {redis.call("get", KEYS[1]), tonumber(ARGV[2]), {KEYS[1]}}
|
||||
"""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_eval_namespaces_keys_and_returns_nested_replies(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
|
||||
assert await transport.eval(_ECHO, ("k",), ("v", 7)) == (b"v", 7, (b"ns:k",))
|
||||
assert await transport.eval(_ECHO, ("k",), ("w", "false")) is None
|
||||
assert await raw.get("ns:k") == b"w"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("decode_responses", (False, True))
|
||||
async def test_eval_preserves_empty_arrays_and_sibling_order(
|
||||
server: fakeredis.FakeServer, decode_responses: bool
|
||||
) -> None:
|
||||
raw = fakeredis.FakeAsyncRedis(server=server, decode_responses=decode_responses)
|
||||
transport = fake_redis_transport(raw, failure_threshold=1)
|
||||
|
||||
assert await transport.eval('return {{}, {1, {"hello", {}}}, {}, 0}', (), ()) == (
|
||||
(),
|
||||
(1, (b"hello", ())),
|
||||
(),
|
||||
0,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_eval_survives_a_script_flush(server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
|
||||
assert await transport.eval(_ECHO, ("k",), ("v", 1)) == (b"v", 1, (b"k",))
|
||||
await raw.script_flush()
|
||||
assert await transport.eval(_ECHO, ("k",), ("w", 2)) == (b"w", 2, (b"k",))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_script_error_is_a_server_failure(server: fakeredis.FakeServer) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
|
||||
failure = await transport.eval('return redis.error_reply("nope")', (), ())
|
||||
|
||||
assert isinstance(failure, RedisFailure)
|
||||
assert failure.kind == "server"
|
||||
assert await transport.eval("return 1", (), ()) == 1
|
||||
|
||||
|
||||
class _OneAtATimeClient:
|
||||
def __init__(self, inner: fakeredis.FakeAsyncRedis) -> None:
|
||||
self.inner = inner
|
||||
self.in_flight = 0
|
||||
self.peak_in_flight = 0
|
||||
|
||||
async def get(self, name: str) -> object:
|
||||
self.in_flight += 1
|
||||
self.peak_in_flight = max(self.peak_in_flight, self.in_flight)
|
||||
await asyncio.sleep(0.01)
|
||||
self.in_flight -= 1
|
||||
return await self.inner.get(name)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_cluster_cache_transport_waits_on_the_connection_gate(raw: fakeredis.FakeAsyncRedis) -> None:
|
||||
with patch("litellm._redis.init_redis_cluster"):
|
||||
cache = RedisClusterCache(
|
||||
startup_nodes=[{"host": "localhost", "port": 7000}],
|
||||
connection_gate=ClusterConnectionGate(limit=1, wait_timeout=5.0),
|
||||
)
|
||||
client = _OneAtATimeClient(raw)
|
||||
litellm.in_memory_llm_clients_cache.set_cache(
|
||||
key=cache._get_async_client_cache_key(), # pyright: ignore[reportPrivateUsage] # seeds the per-loop client slot RedisCache reads
|
||||
value=client,
|
||||
)
|
||||
await raw.set("k", "v")
|
||||
transport = cache.transport()
|
||||
|
||||
replies = await asyncio.gather(*(transport.get("k") for _ in range(5)))
|
||||
|
||||
assert replies == [b"v"] * 5
|
||||
assert client.peak_in_flight == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_replies_stay_bytes_on_a_client_that_decodes_responses(server: fakeredis.FakeServer) -> None:
|
||||
transport = fake_redis_transport(
|
||||
fakeredis.FakeAsyncRedis(server=server, decode_responses=True), failure_threshold=1
|
||||
)
|
||||
|
||||
assert await transport.set("k", "v", ttl=None) is True
|
||||
assert await transport.get("k") == b"v"
|
||||
|
||||
|
||||
async def _settle() -> None:
|
||||
pending: Final = {task for task in asyncio.all_tasks() if task is not asyncio.current_task()}
|
||||
if pending:
|
||||
await asyncio.gather(*pending)
|
||||
await _settle()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _Event:
|
||||
outcome: Literal["success", "failure"]
|
||||
call_type: str
|
||||
target: str | None
|
||||
failed_with: str | None = None
|
||||
|
||||
|
||||
class _RecordingEvents(ServiceLogging):
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.seen: tuple[_Event, ...] = ()
|
||||
|
||||
def has_sinks(self) -> bool:
|
||||
return True
|
||||
|
||||
async def async_service_success_hook(
|
||||
self,
|
||||
service: ServiceTypes,
|
||||
call_type: str,
|
||||
duration: float,
|
||||
parent_otel_span: object | None = None,
|
||||
start_time: datetime | float | None = None,
|
||||
end_time: datetime | float | None = None,
|
||||
event_metadata: object | None = None,
|
||||
caller: str | None = None,
|
||||
) -> None:
|
||||
assert service == ServiceTypes.REDIS and 0 <= duration
|
||||
self.seen = (*self.seen, _Event("success", call_type, current_service_target()))
|
||||
|
||||
async def async_service_failure_hook(
|
||||
self,
|
||||
service: ServiceTypes,
|
||||
duration: float,
|
||||
error: str | Exception,
|
||||
call_type: str,
|
||||
parent_otel_span: object | None = None,
|
||||
start_time: datetime | float | None = None,
|
||||
end_time: float | datetime | None = None,
|
||||
event_metadata: object | None = None,
|
||||
caller: str | None = None,
|
||||
) -> None:
|
||||
assert service == ServiceTypes.REDIS and 0 <= duration
|
||||
self.seen = (*self.seen, _Event("failure", call_type, current_service_target(), str(error)))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_every_command_reports_one_service_event_under_the_callers_target(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
events = _RecordingEvents()
|
||||
transport = fake_redis_transport(raw, failure_threshold=2, events=events)
|
||||
|
||||
with service_target("response_polling"):
|
||||
await transport.set("k", "v", ttl=None)
|
||||
await transport.get("k")
|
||||
server.connected = False
|
||||
failed = await transport.delete("k")
|
||||
|
||||
assert isinstance(failed, RedisFailure)
|
||||
assert events.seen == (
|
||||
_Event("success", "set", "response_polling"),
|
||||
_Event("success", "get", "response_polling"),
|
||||
_Event("failure", "delete", "response_polling", failed.message),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_call_the_breaker_refuses_reports_no_service_event(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
events = _RecordingEvents()
|
||||
transport = fake_redis_transport(raw, failure_threshold=1, events=events)
|
||||
server.connected = False
|
||||
|
||||
await transport.get("k")
|
||||
refused = await transport.get("k")
|
||||
|
||||
assert isinstance(refused, RedisFailure) and refused.kind == "breaker_open"
|
||||
assert tuple(event.outcome for event in events.seen) == ("failure",)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_the_cache_transport_reports_to_the_caches_service_logger(server: fakeredis.FakeServer) -> None:
|
||||
events = _RecordingEvents()
|
||||
transport = fake_redis_cache(server, service_logger=events).transport()
|
||||
await _settle()
|
||||
before = events.seen
|
||||
|
||||
await transport.get("k")
|
||||
server.connected = False
|
||||
await transport.get("k")
|
||||
|
||||
assert tuple(event.outcome for event in events.seen[len(before) :]) == ("success", "failure")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_without_a_service_callback_no_event_is_built(
|
||||
server: fakeredis.FakeServer, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.setattr(litellm, "service_callback", [])
|
||||
service_logger = ServiceLogging(mock_testing=True)
|
||||
transport = fake_redis_cache(server, service_logger=service_logger).transport()
|
||||
await _settle()
|
||||
before = (service_logger.mock_testing_async_success_hook, service_logger.mock_testing_async_failure_hook)
|
||||
|
||||
assert await transport.get("k") is None
|
||||
server.connected = False
|
||||
assert isinstance(await transport.get("k"), RedisFailure)
|
||||
await _settle()
|
||||
|
||||
after = (service_logger.mock_testing_async_success_hook, service_logger.mock_testing_async_failure_hook)
|
||||
assert after == before
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_failure_is_logged_once_with_the_command_and_target(
|
||||
server: fakeredis.FakeServer, caplog: pytest.LogCaptureFixture
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await _settle()
|
||||
server.connected = False
|
||||
caplog.clear()
|
||||
|
||||
with caplog.at_level(logging.WARNING), service_target("response_polling"):
|
||||
failure = await transport.get("k")
|
||||
|
||||
assert isinstance(failure, RedisFailure)
|
||||
assert [record.getMessage() for record in caplog.records] == [
|
||||
f"Redis get failed for response_polling: {failure.message}"
|
||||
]
|
||||
|
||||
|
||||
_Command = Callable[[RedisPyTransport], Awaitable[object]]
|
||||
|
||||
|
||||
_WRONG_TYPE_COMMANDS: Final[tuple[_Command, ...]] = (
|
||||
lambda t: t.incrby("k", 1),
|
||||
lambda t: t.incrbyfloat("k", 1.0),
|
||||
lambda t: t.rpush("k", "x"),
|
||||
lambda t: t.sadd("k", "x"),
|
||||
)
|
||||
_OUTAGE_COMMANDS: Final[tuple[_Command, ...]] = (
|
||||
lambda t: t.incrby("k", 1),
|
||||
lambda t: t.expire("k", timedelta(seconds=1)),
|
||||
lambda t: t.ttl("k"),
|
||||
lambda t: t.lpop("k"),
|
||||
lambda t: t.scan("k", limit=1),
|
||||
lambda t: t.publish("c", "m"),
|
||||
lambda t: t.ping(),
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_counters_add_to_the_stored_value_under_the_namespace(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
|
||||
assert await transport.incrby("n", 2) == 2
|
||||
assert await transport.incrby("ns:n", 3) == 5
|
||||
assert await transport.incrbyfloat("f", 1.5) == 1.5
|
||||
assert await transport.incrbyfloat("f", 0.25) == 1.75
|
||||
|
||||
assert await raw.get("ns:n") == b"5"
|
||||
assert await raw.get("ns:f") == b"1.75"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_expire_sets_the_ttl_in_milliseconds_and_ttl_reads_it_back(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await raw.set("k", "v")
|
||||
|
||||
assert await transport.ttl("k") == -1
|
||||
assert await transport.expire("k", timedelta(milliseconds=1500)) is True
|
||||
assert await transport.expire("absent", timedelta(seconds=5)) is False
|
||||
|
||||
assert 1000 < await raw.pttl("k") <= 1500
|
||||
assert await transport.ttl("k") == 1
|
||||
assert await transport.ttl("absent") == -2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_commands_push_trim_and_pop_from_the_head(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
|
||||
assert await transport.rpush("l", "a", "b", "c", "d") == 4
|
||||
assert await transport.ltrim("l", -3, -1) is True
|
||||
assert await transport.lpop("l") == b"b"
|
||||
assert await transport.lpop("l", 5) == (b"c", b"d")
|
||||
assert await transport.lpop("l") is None
|
||||
assert await transport.lpop("l", 2) is None
|
||||
|
||||
assert await raw.exists("ns:l") == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_lpop_with_a_count_before_redis_7_pops_the_same_items(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = replace(fake_redis_cache(server).transport(), server_version=lambda: 6)
|
||||
await raw.rpush("l", "a", "b", "c")
|
||||
|
||||
assert await transport.lpop("l", 2) == (b"a", b"b")
|
||||
assert await transport.lpop("l", 5) == (b"c",)
|
||||
assert await transport.lpop("l", 5) is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sadd_counts_only_new_members(server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
|
||||
assert await transport.sadd("s", "a", "b") == 2
|
||||
assert await transport.sadd("s", "b", "c") == 1
|
||||
|
||||
assert await raw.smembers("ns:s") == {b"a", b"b", b"c"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_scan_finds_keys_under_the_namespaced_prefix_up_to_the_limit(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
for key in ("ns:team:1", "ns:team:2", "ns:team:3", "ns:user:1", "team:4"):
|
||||
await raw.set(key, "v")
|
||||
|
||||
assert sorted(await transport.scan("team:", limit=10)) == [b"ns:team:1", b"ns:team:2", b"ns:team:3"]
|
||||
assert len(await transport.scan("team:", limit=2)) == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize("command", _WRONG_TYPE_COMMANDS)
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_command_on_the_wrong_type_is_a_server_failure(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis, command: _Command
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await raw.hset("k", "f", "v")
|
||||
|
||||
failure = await command(transport)
|
||||
|
||||
assert isinstance(failure, RedisFailure) and failure.kind == "server"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("command", _OUTAGE_COMMANDS)
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_outage_is_a_transport_failure_for_every_command(
|
||||
server: fakeredis.FakeServer, command: _Command
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
server.connected = False
|
||||
|
||||
failure = await command(transport)
|
||||
|
||||
assert isinstance(failure, RedisFailure) and failure.kind == "transport"
|
||||
|
||||
|
||||
class _ClusterClient:
|
||||
def __init__(self, inner: fakeredis.FakeAsyncRedis) -> None:
|
||||
self.inner = inner
|
||||
|
||||
async def mget(self, keys: Sequence[str]) -> object:
|
||||
raise ResponseError("CROSSSLOT Keys in request don't hash to the same slot")
|
||||
|
||||
async def mget_nonatomic(self, keys: Sequence[str]) -> object:
|
||||
return [await self.inner.get(key) for key in keys]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mget_on_a_cluster_cache_reads_keys_from_different_slots(raw: fakeredis.FakeAsyncRedis) -> None:
|
||||
with patch("litellm._redis.init_redis_cluster"):
|
||||
cache = RedisClusterCache(startup_nodes=[{"host": "localhost", "port": 7000}])
|
||||
litellm.in_memory_llm_clients_cache.set_cache(
|
||||
key=cache._get_async_client_cache_key(), # pyright: ignore[reportPrivateUsage] # seeds the per-loop client slot RedisCache reads
|
||||
value=_ClusterClient(raw),
|
||||
)
|
||||
await raw.set("a", "1")
|
||||
await raw.set("b", "2")
|
||||
|
||||
assert await cache.transport().mget(("a", "b", "c")) == (b"1", b"2", None)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_batch_sends_every_declared_command_in_one_round_trip(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
events = _RecordingEvents()
|
||||
transport = fake_redis_cache(server, namespace="ns", service_logger=events).transport()
|
||||
await _settle()
|
||||
await raw.set("ns:a", "1")
|
||||
before = len(events.seen)
|
||||
batch = transport.batch()
|
||||
|
||||
got = batch.get("a")
|
||||
written = batch.set("b", "2", ttl=timedelta(seconds=30))
|
||||
counted = batch.incrbyfloat("n", 1.5)
|
||||
pushed = batch.rpush("l", "x", "y")
|
||||
read = batch.mget(("a", "b", "missing"))
|
||||
assert not got.done
|
||||
|
||||
assert await got == b"1"
|
||||
assert (await written, await counted, await pushed) == (True, 1.5, 2)
|
||||
assert await read == (b"1", b"2", None)
|
||||
await _settle()
|
||||
assert [event.call_type for event in events.seen[before:]] == ["redis_batch"]
|
||||
assert 0 < await raw.pttl("ns:b") <= 30_000
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_failing_command_in_a_batch_fails_only_its_own_reply(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await raw.hset("h", "f", "v")
|
||||
batch = transport.batch()
|
||||
|
||||
bad = batch.incrby("h", 1)
|
||||
good = batch.incrby("n", 1)
|
||||
|
||||
failure = await bad
|
||||
assert isinstance(failure, RedisFailure) and failure.kind == "server"
|
||||
assert await good == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_an_outage_fails_every_reply_in_the_batch_and_counts_once_on_the_breaker(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_transport(raw, failure_threshold=1)
|
||||
batch = transport.batch()
|
||||
server.connected = False
|
||||
|
||||
declared = (batch.get("a"), batch.incrby("n", 1))
|
||||
replies = [await reply for reply in declared]
|
||||
server.connected = True
|
||||
refused = await transport.get("a")
|
||||
|
||||
assert all(isinstance(reply, RedisFailure) and reply.kind == "transport" for reply in replies)
|
||||
assert isinstance(refused, RedisFailure) and refused.kind == "breaker_open"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_script_redis_does_not_have_cached_still_runs_in_a_batch(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await raw.script_flush()
|
||||
batch = transport.batch()
|
||||
|
||||
scripted = batch.eval("return redis.call('INCRBY', KEYS[1], ARGV[1])", ("n",), (4,))
|
||||
counted = batch.incrby("other", 1)
|
||||
|
||||
assert await scripted == 4
|
||||
assert await counted == 1
|
||||
assert await raw.get("n") == b"4"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_transaction_batch_runs_its_commands_atomically_with_uncached_scripts(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server).transport()
|
||||
await raw.script_flush()
|
||||
batch = transport.batch(transaction=True)
|
||||
|
||||
pushed = batch.rpush("l", "a", "b", "c")
|
||||
trimmed = batch.ltrim("l", -2, -1)
|
||||
scripted = batch.eval("return redis.call('LLEN', KEYS[1])", ("l",), ())
|
||||
|
||||
assert (await pushed, await trimmed, await scripted) == (3, True, 2)
|
||||
assert await raw.lrange("l", 0, -1) == [b"b", b"c"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_a_transaction_stays_one_round_trip_on_a_backend_that_runs_batched_commands_alone(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
events = _RecordingEvents()
|
||||
transport = fake_redis_transport(raw, failure_threshold=1, events=events)
|
||||
alone_backend = replace(transport.batch().engine.backend, runs_alone=True)
|
||||
batch = RedisPyCommandBatch(transport, PipelineBatch(alone_backend, name="tx", transaction=True))
|
||||
|
||||
pushed = batch.rpush("l", "a", "b")
|
||||
trimmed = batch.ltrim("l", -1, -1)
|
||||
|
||||
assert (await pushed, await trimmed) == (2, True)
|
||||
await _settle()
|
||||
assert [event.call_type for event in events.seen] == ["tx"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_request_batch_commands_share_the_round_trip_of_redis_batch_callers(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
cache = fake_redis_cache(server, namespace="ns")
|
||||
await raw.set("ns:legacy", '"old"')
|
||||
await raw.set("ns:moved", "new")
|
||||
transport = cache.transport()
|
||||
assert transport.request_batch() is None
|
||||
|
||||
with request_redis_batch_scope():
|
||||
legacy = active_request_redis_batch(cache)
|
||||
commands = transport.request_batch()
|
||||
assert legacy is not None and commands is not None
|
||||
old_read = legacy.mget(["legacy"])
|
||||
new_read = commands.get("moved")
|
||||
missing = commands.get("absent")
|
||||
|
||||
assert await old_read == {"legacy": "old"}
|
||||
assert await new_read == b"new"
|
||||
assert await missing is None
|
||||
assert legacy.flushes == 1
|
||||
assert legacy.read_as_missing("absent") is True
|
||||
assert legacy.read_as_missing("moved") is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_publish_reaches_a_subscriber_on_the_namespaced_channel(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_cache(server, namespace="ns").transport()
|
||||
subscription = await transport.subscribe("config")
|
||||
assert not isinstance(subscription, RedisFailure)
|
||||
|
||||
assert await transport.publish("config", "one") == 1
|
||||
assert await transport.publish_all("config", ("two", "three")) == (1, 1)
|
||||
assert await raw.publish("config", "elsewhere") == 0
|
||||
|
||||
received = [await subscription.get_message(timeout=1) for _ in range(4)]
|
||||
messages = [message for message in received if message is not None]
|
||||
assert messages == [
|
||||
PubSubMessage(channel=b"ns:config", data=b"one"),
|
||||
PubSubMessage(channel=b"ns:config", data=b"two"),
|
||||
PubSubMessage(channel=b"ns:config", data=b"three"),
|
||||
]
|
||||
assert await subscription.close() is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_health_and_admin_commands_still_run_while_the_breaker_is_open(
|
||||
server: fakeredis.FakeServer, raw: fakeredis.FakeAsyncRedis
|
||||
) -> None:
|
||||
transport = fake_redis_transport(raw, failure_threshold=1)
|
||||
await raw.set("k", "v")
|
||||
server.connected = False
|
||||
await transport.get("k")
|
||||
server.connected = True
|
||||
assert isinstance(refused := await transport.get("k"), RedisFailure) and refused.kind == "breaker_open"
|
||||
|
||||
assert await transport.ping() is True
|
||||
clients = await transport.client_list()
|
||||
assert not isinstance(clients, RedisFailure) and len(clients) >= 1
|
||||
assert await transport.flushall() is True
|
||||
assert await raw.exists("k") == 0
|
||||
|
||||
|
||||
class _InfoClient:
|
||||
async def info(self) -> object:
|
||||
return {"redis_version": "7.4.0", "connected_clients": 3, "db0": {"keys": 2, "expires": 0}}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_info_returns_the_parsed_sections(raw: fakeredis.FakeAsyncRedis) -> None:
|
||||
transport = replace(fake_redis_transport(raw, failure_threshold=1), client=lambda: _InfoClient())
|
||||
|
||||
info = await transport.info()
|
||||
|
||||
assert not isinstance(info, RedisFailure)
|
||||
assert (info["redis_version"], info["connected_clients"], info["db0"]) == ("7.4.0", 3, {"keys": 2, "expires": 0})
|
||||
|
|
@ -25,7 +25,7 @@ _REPO: Final = Path(__file__).resolve().parents[2]
|
|||
|
||||
_REDIS_PRODUCER_ROOTS: Final = ("litellm", "enterprise")
|
||||
# The cache implementations and facades: they emit the service events, their callers declare the family.
|
||||
_CACHE_LAYER_DIRS: Final = ("litellm/caching", "litellm/_v2/cache")
|
||||
_CACHE_LAYER_DIRS: Final = ("litellm/caching", "litellm/_v2/cache", "litellm/redis/transport")
|
||||
# Helpers that act on a cache handed in by the declaring caller, or forward to the response-cache facade.
|
||||
_CACHE_PARAMETER_HELPERS: Final = frozenset(
|
||||
{
|
||||
|
|
|
|||
72
uv.lock
generated
72
uv.lock
generated
|
|
@ -2123,6 +2123,11 @@ wheels = [
|
|||
{ url = "https://files.pythonhosted.org/packages/49/b5/82f89307d0d769cd9bf46a54fb9136be08e4e57c5570ae421db4c9a2ba62/fakeredis-2.34.1-py3-none-any.whl", hash = "sha256:0107ec99d48913e7eec2a5e3e2403d1bd5f8aa6489d1a634571b975289c48f12", size = 122160, upload-time = "2026-02-25T13:17:49.701Z" },
|
||||
]
|
||||
|
||||
[package.optional-dependencies]
|
||||
lua = [
|
||||
{ name = "lupa" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fastapi"
|
||||
version = "0.136.3"
|
||||
|
|
@ -5148,7 +5153,7 @@ dev = [
|
|||
{ name = "basedpyright" },
|
||||
{ name = "botocore-stubs" },
|
||||
{ name = "diff-cover" },
|
||||
{ name = "fakeredis" },
|
||||
{ name = "fakeredis", extra = ["lua"] },
|
||||
{ name = "fastapi-offline" },
|
||||
{ name = "hypothesis" },
|
||||
{ name = "keyring" },
|
||||
|
|
@ -5377,7 +5382,7 @@ dev = [
|
|||
{ name = "basedpyright", specifier = "==1.39.7" },
|
||||
{ name = "botocore-stubs", specifier = "==1.43.14" },
|
||||
{ name = "diff-cover", specifier = "==9.7.2" },
|
||||
{ name = "fakeredis", specifier = "==2.34.1" },
|
||||
{ name = "fakeredis", extras = ["lua"], specifier = "==2.34.1" },
|
||||
{ name = "fastapi-offline", specifier = "==1.7.6" },
|
||||
{ name = "hypothesis", specifier = "==6.165.10" },
|
||||
{ name = "keyring", specifier = "==25.7.0" },
|
||||
|
|
@ -5602,6 +5607,69 @@ wheels = [
|
|||
{ url = "https://files.pythonhosted.org/packages/eb/cf/e4e016820516c1f0c85549e356865c2319cf6afba8ab386611b1d03cf2b6/lunary-1.4.37-py3-none-any.whl", hash = "sha256:7289330e851a481404c92213ce480c0c7be9bfa56982a1385e08e3665abefe05", size = 25581, upload-time = "2026-02-12T08:15:03.892Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "lupa"
|
||||
version = "2.8"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c3/a6/0f869fbb07c393f15473b1eefefb7b5bec162fb7481803d040ed4dc46002/lupa-2.8.tar.gz", hash = "sha256:d8022641b9ec8ecf2c5ecbe9f47e5a70e0b87c4b5ae921b92cb02a638e0acd08", size = 6156370, upload-time = "2026-04-15T20:08:30.534Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/09/21/9be4516ddd22f8eadba336d9ba065d17d79108465ae1b7f71424ab99b9d0/lupa-2.8-cp310-abi3-win32.whl", hash = "sha256:c2a5fd15dc62374e1661a55f01744c9ec1c56f291ba4a0749d3af2174556e78f", size = 1594887, upload-time = "2026-04-15T20:05:23.377Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2d/99/1557c9685d7034d9ce8dd2b54c40a26d6deb7c67c1fdb5c801abd1a02c3f/lupa-2.8-cp310-abi3-win_arm64.whl", hash = "sha256:9e304fb1c50cf23fd8882afbe1aa87525ef8a72667bcab3b37b2bbb2bc542269", size = 1371742, upload-time = "2026-04-15T20:05:27.417Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1c/34/05ce4745b191633f90ff1ab50f1a19a37da282bb0a41fb500d9157fc9b8f/lupa-2.8-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:97bd01e90b8031e56a5fd5bb70605aea09f1dba675c1140308a52780f93d06f1", size = 1202714, upload-time = "2026-04-15T20:05:31.088Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7d/d2/f70fdbeec2d4c69ee6a469e6cddde9635fff4af4e13fb652e6a1229eef51/lupa-2.8-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b5ebe1a13c45767919c86750b84fe2da9f6288b6f3cea4ce7660bb2abc9d921", size = 1857453, upload-time = "2026-04-15T20:05:34.611Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/97/dc/6fcda0e36e75eb6cb98dc9190fa4737d727eeae29e58f892980b2c96b656/lupa-2.8-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:097e7d0f1719a88020b67c82e05d53d7973c166952393afcecfd8434c7e19a15", size = 2408890, upload-time = "2026-04-15T20:05:37.994Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/58/29/7ea176eac3c1dac83d059762daa875ad1390decc0bf2c3b4c7bbfc1f1665/lupa-2.8-cp310-cp310-win_amd64.whl", hash = "sha256:7bb223ee8f72d0dc076b0d65296ee72f1c69450f9d2fed5315f7707d98c4a03d", size = 1910396, upload-time = "2026-04-15T20:05:41.163Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/0a/5a740717f27aa77481e6a61b97cf79d1e0c1ede729b1268caacded915326/lupa-2.8-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b12e43c1fb787189dfc28cd604aef0baa2cb95e27da19498d520361d0ace070a", size = 1202376, upload-time = "2026-04-15T20:05:44.049Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1b/75/6b64d0098c64275a801896cb7a6a30e7e653d25fa102c64e747292afcdbb/lupa-2.8-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f6f603391dffb256e36a79fd2044084d5f4b8a0a4c0e5ad291cd3ab3aaf1fd0a", size = 1839271, upload-time = "2026-04-15T20:05:47.399Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/2f/0d4f00563046ff616ef6a421f8b776a5ffb327f7b32ed69e856d52b917a8/lupa-2.8-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9f6f41c91366e7d0d474f87d81c1274af861f40812bf729c9f97ab4c8f3c7ac8", size = 2376251, upload-time = "2026-04-15T20:05:49.891Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4c/8e/caa83237f427d9e85b7f02c816e7270c9c9571dec1673e06b0180402f70e/lupa-2.8-cp311-cp311-win_amd64.whl", hash = "sha256:f5a6af145b0ea818f01d27bfe2583a4b538570bef61d22c8773e0eccf011234c", size = 1923488, upload-time = "2026-04-15T20:05:52.954Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ad/0b/368f2f0bc750b25c69d4563e44f677925ab5dd3d2887f9b0c15465d21a2a/lupa-2.8-cp312-abi3-macosx_10_13_x86_64.whl", hash = "sha256:f4342f4de76ae7ce2ab0672d36003bdb7e1a33252f293b569298ddd792e70e33", size = 1194056, upload-time = "2026-04-15T20:05:55.794Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5b/0f/c89eb8dd36fdea4e50ae3f7f5275bea3b0cc5d4057b8ee7b3bbc78010422/lupa-2.8-cp312-abi3-manylinux2010_i686.manylinux_2_12_i686.manylinux_2_28_i686.whl", hash = "sha256:4203fa1659315e939a5304e75001b8cc14234fb3cbb3ed86c049b0cc5d90fcee", size = 1434278, upload-time = "2026-04-15T20:05:57.94Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/47/30/c3b4d2cd8733621b404b8a4214e5f852955c4ba632546dc84123bea9ee89/lupa-2.8-cp312-abi3-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:81f2d843ce668b653146c007467570210ae44be51dac6926666c51d49536f307", size = 1150068, upload-time = "2026-04-15T20:06:01.04Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8d/d2/bac12c398519efafc6af84be1974edd0d7a4895fb4735b5c8d615d298595/lupa-2.8-cp312-abi3-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d3d0cde2c77588d1c60875a4f34f059513476c6e1775351897195b51e0f3df08", size = 1409532, upload-time = "2026-04-15T20:06:03.592Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9c/6a/18b52e11962014026e07813530b0b108ee8bc0a2a13ef0eaea5d41dce023/lupa-2.8-cp312-abi3-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9e0d11b8f3a8dac6413f704fef7161d048bb10c58bdac6cbffa5e60efa56e9a3", size = 1242687, upload-time = "2026-04-15T20:06:06.863Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b3/8e/7fd4eb049875f61429b96780d2eae4700f0e78fe0a52db8edb231b1cd09f/lupa-2.8-cp312-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:54cff414f21f8cd8c6be4aae52541f3b9cd39602b59e3a3db9b5c9f9f674ff18", size = 1856038, upload-time = "2026-04-15T20:06:09.358Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/f9/37ad9d2773d30f2931890d310a4bdce28d45484206e6f48bc18b0325eabd/lupa-2.8-cp312-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:24b4d8af5558e549b70daf1547f5c1c1d664ecea9fc790f83efe5d75e9a93797", size = 1128982, upload-time = "2026-04-15T20:06:12.312Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/57/31/c0fd7984c24844ea79caa45c0235f61a06b38fd69a839f6c62770f8d684a/lupa-2.8-cp312-abi3-musllinux_1_2_i686.whl", hash = "sha256:ce86dff1ee7f7cf45f5622065ae991949dd7bb1703581cbc58a630137bb7ccf9", size = 1457594, upload-time = "2026-04-15T20:06:15.881Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/11/f5/a28e411be30ec1bf0db1eb0c087eebc73be9e7a1adcfe6ac209861ccc446/lupa-2.8-cp312-abi3-musllinux_1_2_ppc64le.whl", hash = "sha256:f4d01b2a08c70bbb883a9e082b6b36b89121ed5910b710f1ba11c73295ff4fba", size = 1425721, upload-time = "2026-04-15T20:06:18.009Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ed/c1/359f767c4ae024be30d909fe8a9f0e9af266bad47ce2bd2ed248fb986fcf/lupa-2.8-cp312-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:7f210d5a8353e510ea1199c42cf3cbdd630553bf2bc8fb4c00fea06fdec7c798", size = 1253258, upload-time = "2026-04-15T20:06:21.17Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/17/52/473f11790c261fd02bbf318a546fe040e9ec9f677181272fa78d3b4112a4/lupa-2.8-cp312-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:4f81a02806e7c7ad26d8c6fa222c8bef1b0c1b124347c879be880b41339d41e4", size = 2395272, upload-time = "2026-04-15T20:06:24.137Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/94/bf/75c8795655a8836eab6a11a630352c4b7c5dc5c54d075077bc9bffdeee45/lupa-2.8-cp312-abi3-win32.whl", hash = "sha256:360056453a7a4eaa4ac5a204c31a5a014b1eb2ee5490603234d2ba831684f1f2", size = 1606136, upload-time = "2026-04-15T20:06:27.815Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d8/29/11a2cdd612b6f55e506292dfb6ba343216e80a693e7fe3f876ef204ce9c6/lupa-2.8-cp312-abi3-win_arm64.whl", hash = "sha256:1628371c6592a6d5650497a9e31fb2bb3a7e9883c1f301d1111265e484045af9", size = 1364495, upload-time = "2026-04-15T20:06:30.254Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4d/17/fa834b6b09ad17e7df5d0f7715d64877a125a3776ada689751a1f9dc2959/lupa-2.8-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:450650f91c48c2415b0d59ab3abfcfda3b6efb5b858205f4d4bda8ad141fa529", size = 1190111, upload-time = "2026-04-15T20:06:32.84Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ab/43/45589901b7d1a0e3a9d91d19a311fb6a56924e8571536c3f2212160fd953/lupa-2.8-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:27044f3363047f946b3d3aab9157cbd172b3538ada9ec1baef43432bf7d03a78", size = 1812999, upload-time = "2026-04-15T20:06:35.664Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/ac/4ade7d15ff5c61758d7943ac6f0a496bf1cc65b6c09f842b52a0702e664c/lupa-2.8-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8cf4f064a0e5531afce2d7d750120c10c10f9529139af6ca6150d13151034398", size = 2368731, upload-time = "2026-04-15T20:06:37.959Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0c/27/05f950d15b8ab120b39c43588b438ff3ace70c1b1b0225a960393a497483/lupa-2.8-cp312-cp312-win_amd64.whl", hash = "sha256:281bedc5deb92d31e649a3552edd662449365a635904fa4d5cb4509c7245e34e", size = 1941809, upload-time = "2026-04-15T20:06:40.302Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a6/3f/19f83c3a0c84dc8bea8a58e7416dca6a3ede662c33c8d1ec758e5afc754a/lupa-2.8-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45fc9da0145ecb0083ef5ff9975116cc784bd0258bdc2bd131ba15483ce18398", size = 1201203, upload-time = "2026-04-15T20:06:42.169Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/89/0f/a14f0073f09610158038582e230618a48c14da6bd88185289461aa4cb854/lupa-2.8-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:58e18afed57955b41130e269c78f53d4123ab86e236b53816f4cbffa25cb5d30", size = 1806210, upload-time = "2026-04-15T20:06:45.486Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2f/14/48fff156c63a136001a7620878af7d31aa07e66b495ed621e3eddd73c294/lupa-2.8-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fc47f536ac13a79cef47d29a2b205576a22841f042a2bcec1676b95806e7706a", size = 2359005, upload-time = "2026-04-15T20:06:47.819Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fe/18/3ac638ec90edf178242b8a2b2f00f8adae694248c03a26341ef941bb746e/lupa-2.8-cp313-cp313-win_amd64.whl", hash = "sha256:ce9404c661dbac65cc9bed351ad45e797af93d30d70be309a3fa8209ac86d93b", size = 1936754, upload-time = "2026-04-15T20:06:50.448Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b0/ef/5ee5fed6ea7459a671196359ce04bfeeaf26be1dac8ff24bf28e5c7a6e81/lupa-2.8-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:348c3f8ecabb6324dcbc05c2740d762ef8fcec7b06c79e45262ab97a217684e3", size = 1209388, upload-time = "2026-04-15T20:06:53.022Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6e/b1/67a940d5542cb0384b443fe951b5a83ea9340d1333a733a258fdd1c619ba/lupa-2.8-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:951496471056061598a7d1729a6cdf48d662fec777a9f2d8aa5a1e62fd30e5a5", size = 1826821, upload-time = "2026-04-15T20:06:55.699Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/a2/b354e5ba3b911ec50686003dc8897e892b9e8c5c036b33219b03d54c4daf/lupa-2.8-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a591b9947ca347b41a63370e121d6e2b1458fe6dde9ae065029ec10a37f25ff4", size = 2366893, upload-time = "2026-04-15T20:06:58.9Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8e/52/d76066401f29539df5352f70ecded66576f32933b6045cd0bfc56cb770b9/lupa-2.8-cp314-cp314-win_amd64.whl", hash = "sha256:3903c9cf628dae2f56405503247b77a61a3a61bd2dda470e336950c74776d55d", size = 1994716, upload-time = "2026-04-15T20:07:19.194Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c3/bd/3efc437a4361c16d25e66478c50357c9a8e8ecfb718fe749eb9ca3176ef6/lupa-2.8-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f711a8ab0486b9ac6fdda94a22ddcfbc9f0d4a27e3a8cf1bf79c6e48b33017c1", size = 1251217, upload-time = "2026-04-15T20:07:01.64Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ea/f4/2e9f8ecbaca854bfdf14af8a9b505ec0cbc640377b3b218921594b7563cd/lupa-2.8-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:dc51250e76367a3e27fcd01dc769b9bfcbbc34f48df48dde53d6af6e75b7eaa5", size = 1814701, upload-time = "2026-04-15T20:07:04.149Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ba/53/4000b1acaa8b1f3827fcff0cfcdff44d3befddda42cab7e685a49689b5a1/lupa-2.8-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f8a22088a552828958603323f0a5c4b3e11e03b75d0bf4c965ef879de9b60a8d", size = 2348414, upload-time = "2026-04-15T20:07:07.285Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/78/26ee48d3890cddf03cefb65f433e3492759c0b3c0582180755bddbaab7bd/lupa-2.8-cp314-cp314t-win32.whl", hash = "sha256:4f7c553c1d8cfffbe85d81daef730d12cae4b6002d457542914da0ac8a1145b3", size = 1831611, upload-time = "2026-04-15T20:07:09.752Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3c/d1/4a5cc64a3cad22821ae4c3f7a90456a08ca19457d8354f4abf46ad03c7e8/lupa-2.8-cp314-cp314t-win_amd64.whl", hash = "sha256:d8766aff03a78c80ad2d188a8bdb216de5ec838359cd87e05bbdfa56394a6105", size = 2209250, upload-time = "2026-04-15T20:07:11.906Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/7c/cdcb654daf668192aaf36b0aeb94f2281dad092aaa5003688691131736ea/lupa-2.8-cp314-cp314t-win_arm64.whl", hash = "sha256:91d622777febda3ab1bed1d45295f2f32a4680c7b3d7caf8c669998ed5c44118", size = 1126735, upload-time = "2026-04-15T20:07:15.434Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1d/44/de1961ad38e17cd326a53c246c7e3b91178ed578f4cf22ffcd5e7e11b041/lupa-2.8-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:b036738282a5acd2e71fdddb317c9df8b87c1673aa57f403d05fcc2be8abc4ba", size = 1186020, upload-time = "2026-04-15T20:07:35.017Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/13/c2/276f0b9dc8bcc5a8a58af5316dfa0e6f56be3613dd6dbcc8d3d2cb6559ba/lupa-2.8-cp39-abi3-manylinux2010_i686.manylinux_2_12_i686.manylinux_2_28_i686.whl", hash = "sha256:ac6b6e8d0e617e26a98cbb44880bcd75de5d32b3ad7b3b3793583909292b47ed", size = 1468944, upload-time = "2026-04-15T20:07:37.782Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/63/38/52934e52a5180dc6425d20284d004fe4b27a4f9171a82dc99fb67af250bf/lupa-2.8-cp39-abi3-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:ba3a7dd839f90c3d2e53bebe3c192b1f3f9fd720a6781256405123211fd0dce6", size = 1172998, upload-time = "2026-04-15T20:07:40.812Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c7/82/76b3809bd0839d9b3b4ec58d06591e08f17337b6d9576877cb9d48b34e94/lupa-2.8-cp39-abi3-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d7edb13a7a5250b5c6c22d1495d9e842b5c9fc5081c8fe6b5efe2112fe3e41f9", size = 1449975, upload-time = "2026-04-15T20:07:44.262Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/16/07/2f89d54f747c67c23b4b9ae4aa8c8dd06bb409155dedcf406157f2736b66/lupa-2.8-cp39-abi3-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:891f72e0bffbed1e4175f975aeb2a083956586a100066525e1be485f617f7b25", size = 1281944, upload-time = "2026-04-15T20:07:46.458Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/bd/7375d2b0fcae79d806baf52a76f26c96964593f58e1372d13ae5ac09c676/lupa-2.8-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:a295f87b5b7ebbfd5191932e8cb0e51df3c7769101ac6b6c7d7c9fb27bfd1307", size = 1910455, upload-time = "2026-04-15T20:07:49.75Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/0c/8abb3bc0e08b311fc01db05b6e9f9ff31a8f65e4fc3f0aeb05cfef75c8ac/lupa-2.8-cp39-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:4fe5d7a810b64ea8511eb885fc8cdde042ee5ff7b7d08ae78f32449756acb177", size = 1155548, upload-time = "2026-04-15T20:07:52.657Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/80/2e/9eeecd3f493099721c1d3f31beeca23a4237db1a54223684df4dc96aa1bd/lupa-2.8-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:bfc470012ef66ad064c7bd77416af03a3452ef630b04b9012595ea13f2e54518", size = 1489232, upload-time = "2026-04-15T20:07:54.92Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c3/13/731c99dc2e7652ae818a6de45bdf0142049f7cb566049061c898355f1891/lupa-2.8-cp39-abi3-musllinux_1_2_ppc64le.whl", hash = "sha256:250e035fdaffe8c87093e3ebc206ac29a26131b1568ea711d780c26001ce96e7", size = 1466321, upload-time = "2026-04-15T20:07:57.627Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/de/71/3ad8cc4fc05a77dc0d3f7079348bd1cad4675a0d14c24f8e6a3ce5f008f7/lupa-2.8-cp39-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:b9bddb09acfffb4f828f790f444b11dc0cca591afea1a244d9329eea2d20c003", size = 1288577, upload-time = "2026-04-15T20:07:59.913Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d8/b2/1175f6d0aa7b68627fbe2f58bd1e8bea36a89d10dfd67671d2b024c96162/lupa-2.8-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:2e64acbbd47e9b82a64405a39e0d2b36a5a7dad8ab41c0f3437f572f7d282ba3", size = 2444866, upload-time = "2026-04-15T20:08:02.753Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/92/f7/e78df680c7a0ea452daac07467ca188d63c2c00ca1c884c0a50e27eb83b5/lupa-2.8-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:32e4e5103bbddcdd2458fb2ccae6c8ba11c9997c711d7e379e0d45551d109c76", size = 1778509, upload-time = "2026-04-15T20:08:21.784Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e6/23/0e53cabb16b2a8aa9cf1fde499c097d8942c5dab709fc8e921f3b824b18b/lupa-2.8-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7667001804657496dee9feced2daae5000b4604a3218dd8e6b7b754982ba88b8", size = 2300480, upload-time = "2026-04-15T20:08:24.394Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/85/0271227eab939921a12ebba5d17aa4cd18346aa534ca7f5da09cd0b63dd4/lupa-2.8-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:86f6f668966965b15247dc32d064cfe7be67b71e584ccfacbe2f637575296878", size = 1847445, upload-time = "2026-04-15T20:08:27.031Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "lxml"
|
||||
version = "6.1.1"
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue