scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102

Flush virtual-key model max budget increments to Redis after success logging.

Browse source 
_PROXY_VirtualKeyModelMaxBudgetLimiter subclasses RouterBudgetLimiting but does not run its __init__, so the periodic Redis sync task never starts and spend stayed in memory. Push the increment pipeline when Redis is configured so multi-worker enforcement and cache keys stay consistent.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Milan 2026-05-07 01:20:03 +03:00
parent bd1a05aed9
commit 32b031aa2c
No known key found for this signature in database
1 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
litellm/proxy/hooks/model_max_budget_limiter.py
View file

@ -319,6 +319,9 @@ class _PROXY_VirtualKeyModelMaxBudgetLimiter(RouterBudgetLimiting):
response_cost=response_cost,
)
if self.dual_cache.redis_cache is not None:
await self._push_in_memory_increments_to_redis()
verbose_proxy_logger.debug(
"current state of in memory cache %s",
json.dumps(