* fix(proxy): stop update_cache from resurrecting evicted auth cache entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): update update_cache pins for synchronous write-back
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): wrap long assert in update_cache tag test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover refill race and pin TTL with injected clock
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): count global proxy spend for every request with a user, cached on this worker or not
* fix(proxy): add global proxy spend with an atomic Redis increment that never re-creates a reset counter
The global proxy spend bump read the total, awaited, and wrote old + cost back
from a background task, so a budget reset that deleted the key in between got
the pre-reset total written back, and workers overwrote each other's totals.
RedisCache.async_increment_if_present runs INCRBYFLOAT only when the key
exists, keeping its TTL, and this worker's copy is then set from the shared
total with no await between its read and write. The bump now runs whenever a
proxy max_budget is set, matching the DB writer, and the armed post-call read
and the batch-read declaration it used are removed.
* fix(proxy): report the global spend Redis increment to service metrics and request traces
The global proxy spend increment now emits Redis service success and
failure events under the spend_counters target and passes the request
span through, so it shows as redis.incr spend_counters like the write
it replaced.
The unit tests run the real RedisCache increment against fakeredis
instead of mocks, and a new test checks the span lands under the
request span on success and on a Redis failure.
* test(proxy): keep the spend counter Redis fake off the network
* test(proxy): count increment_if_present as a shared-cache call in the key-family guard
* fix(proxy): keep the newer global spend total when Redis replies arrive out of order
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): refresh cached team on /team/block and /team/unblock
Both endpoints wrote `blocked` to Postgres and returned without touching the
management-object cache. Auth resolves teams through `team_id:{team_id}`
before reading the DB and `common_checks` enforces `blocked` off that cached
object, so a team whose traffic had already warmed the cache kept serving
requests after being blocked, and kept getting rejected after being
unblocked, until the entry expired.
`_refresh_cached_team` already exists for exactly this and its docstring says
every endpoint mutating `litellm_teamtable` must call it; these two were the
only mutating endpoints that didn't. The update now also asks for
`object_permission` so the refreshed cache entry doesn't drop the relation,
matching `team_model_add` and `team_model_delete`.
* fix(proxy): keep a team block from being undone by in-flight spend writes
update_cache read the cached team when a request finished and wrote that
whole copy back in a background task, so a team blocked or invalidated
while requests were in flight came back unblocked on every busy worker
until its 60 s cache entry expired. The team spend now goes onto the team
held in this worker's memory at write time, and a missing entry is left
missing for the next auth read to load.
Block and unblock responses keep litellm_model_table and object_permission
null, as before this PR, while the cache still gets both relations.
* test(proxy): type the team spend cache test parameters
* test(proxy): hold a team block under in-flight team spend and keep team aliases routing after block and unblock
---------
Co-authored-by: Hamza Shah <hamzasepal@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): move proxy_server, _experimental and db tests into tests/unit/proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): keep tuple identity in proxy state restore and fix misc target paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 18:14:24 +00:00
Renamed from tests/test_litellm/proxy/proxy_server/test_spend_counters.py (Browse further)
