litellm/tests/unit/test_router
devin-ai-integration[bot] ffb15f946f
perf(proxy): one request-scoped Redis pipeline for auth, spend, rate-limit and routing reads (#43407)
RedisBatch: one pipeline per Redis backend for independently declared operations (MGET, GET, Lua
scripts, INCRBYFLOAT, SET, DEL), a future per operation so each owner keeps its own fallback, Redis
Cluster hash-slot fallback. A request-scoped batch middleware shares that pipeline across the auth
identity reads and write-back, the spend counter MGET, the rate limiter Lua groups and the routing
read. A rate-limit denial stands when another pipelined group fails; every pipelined group is refunded
on rejection; local cooldowns win over the prefetch.

The routing prefetch failure log line strips request line breaks (CodeQL py/log-injection)

Resolves LIT-8882

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-29 16:42:13 -07:00
..
__init__.py test: add __init__.py to every tests/unit directory phase 16 touches 2026-09-20 11:52:38 +00:00
test_enforce_model_rate_limits.py test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
test_io_token_rate_limits.py test: migrate phase 16 legacy tests to tests/unit 2026-09-20 10:59:02 +00:00
test_router.py perf(proxy): one request-scoped Redis pipeline for auth, spend, rate-limit and routing reads (#43407) 2026-09-29 16:42:13 -07:00